lOActive 


Reverse Engineering Code with 

IDA Pro 


Uncover the Good, the Bad, and the Ugly Code with IDA Pro! 

• Master the Most Powerful Disassembler and Debugger for Windows, 

Linux, or OSX 

• Single-Step through Code to Understand the Complexities of Worms, 
Viruses, and Trojans 

• Automate Even the Most Complex Tasks using the IDC Scripting Language 


Dan Kaminsky, Technical Editor and Contributing Author 

Justin Ferguson 
Jason Larsen 
Luis Miras 
Walter Pearce 

WWW 





Reverse Engineering 
Code with IDA Pro 



www.allitebooks.com 


This page intentionally left blank 


www.allitebooks.com 


Elsevier, Inc., the author(s), and any person or firm involved in the writing, editing, or production (collectively 
“Makers”) of this book (“the Work”) do not guarantee or warrant the results to be obtained from the Work. 

There is no guarantee of any kind, expressed or implied, regarding the Work or its contents. The Work is 
sold AS IS and WITHOUT WARRANTY. You may have other legal rights, which vary from state to state. 

In no event will Makers be liable to you for damages, including any loss of profits, lost savings, or other 
incidental or consequential damages arising out from the Work or its contents. Because some states do not 
allow the exclusion or limitation of liability for consequential or incidental damages, the above limitation 
may not apply to you. 

You should always use reasonable care, including backup and other appropriate precautions, when working 
with computers, networks, data, and files. 

Syngress Media®, Syngress®, “Career Advancement Through Skill Enhancement®,” “Ask the Author 
UPDATE®,” and "Hack Proofing®,” are registered trademarks of Elsevier, Inc. “Syngress: The Definition of 
a Serious Security Library”™, “Mission Critical™,” and “The Only Way to Stop a Hacker is to Think Like 
One™” are trademarks of Elsevier, Inc. Brands and product names mentioned in this book are trademarks 
or service marks of their respective companies. 


KEY 

SERIAL NUMBER 

001 

HJIRTCV764 

002 

P09873D5FG 

003 

829KM8NJH2 

004 

BAL923457U 

OOS 

CVPLQ6WQ23 

006 

VBP965T5T5 

007 

HJJJ863WD3E 

008 

2987GVTWMK 

009 

629MP5SDJT 

010 

IMWQ295T6T 


PUBLISHED BY 
Syngress Publishing, Inc. 

Elsevier, Inc. 

30 Corporate Drive 
Burlington, MA 01803 

Reverse Engineering Code with IDA Pro 

Copyright © 2008 by Elsevier, Inc. All rights reserved. Printed in the United States of America. Except as 
permitted under the Copyright Act of 1976, no part of this publication may be reproduced or distributed 
in any form or by any means, or stored in a database or retrieval system, without the prior written 
permission of the publisher, with the exception that the program listings may be entered, stored, and 
executed in a computer system, but they may not be reproduced for publication. 

Printed in the United States of America 
1234 5 67890 

ISBN 13: 978-1-59749-237-9 


Publisher: Andrew Williams 
Technical Editor: Dan Kaminsky 
Project Manager: Anne McGee 
Page Layout and Art: SPi 

For information on rights, translations, and bulk sales, contact Matt Pedersen, Commercial Sales Director 
and Rights, at Syngress Publishing; email m.pedersen@elsevier.com. 


www.allitebooks.com 


This page intentionally left blank 


www.allitebooks.com 


About lOActive 


Established in 1998, lOActive has successfully positioned itself as an industry 
leader in the Northwest’s computer security community, where it specializes 
in infrastructure assessment services, apphcation security services, managed 
services, incident response services, and education services. The company 
has helped various Fortune 500 organizations with services ranging from 
enterprise risk management to independent technical validations of security 
hardware and a wide range of apphcations. It has also been commissioned 
to work on IT disaster recovery and business continuity planning for major 
insurance companies, state organizations and energy companies. lOActive s 
consultants are members and active contributors to local and nationahy 
recognized computer security organizations such as SANS, Agora, CRIME, 
ISSA, CTIN,WSA, EIoneyNet Research Alliance, OWASP, and the University 
of Washington Information Assurance School. 


v 


www.allitebooks.com 


Technical Editor and 
Contributing Author 


Dan Kaminsky is the Director of Penetration Testing for IOActive. Previously of 
Cisco and Avaya, Dan has been operating professionally in the security space since 
1999. He is best known for his “Black Ops” series of talks at the well respected Black 
Hat Briefings conferences. He is also the only speaker who has attended and spoken at 
every single “Blue Hat” Microsoft internal training event. Dan focuses on design level 
fault analysis, particularly against massive-scale network applications. Dan regularly 
collects detailed data on the health of the worldwide Internet, and recently used this 
data to detect the worldwide proliferation of a major rootkit. Dan is one of the few 
individuals in the world to combine both technical expertise with executive level 
consulting skills and prowess. 


www.allitebooks.com 


Contributing Authors 


Justin Ferguson is a security consultant and researcher at IOActive. He is 
involved with helping Fortune 500 companies understand and mitigate risk 
introduced in complex software computing environments via the Application 
Security Practice at IOActive. Justin has over six years experience working 
as a reverse engineer, source code auditor, malware analyst, and enterprise 
security analyst for industries ranging from financial institutions to the 
federal government. 

I would like to thank my father, Bruce Dennis Ferguson, who was a great man; 

I regret never having apologized to you nor allowing you to see the man your son 
has become. I would like to thank all of the blue collar union workers from Boston 
who worked themselves to the bone to make sure their children had a better life. No 
mention o f these men would be complete if I neglected the women who stood by their 
sides and saw them through each day; you all truly are beautiful. I’d like to take 
a moment to remember everyone from the South End and Brockton/South Shore 
who didn’t make it and for those still struggling; continue on with the belief that 
unearned suffering is redemptive. Saint Jude, pray for us all. 

Jason Larsen has penetrated and owned some of the most integral systems 
on the planet. His career began when he was at Idaho State University and 
detected Internet-wide stealth scanning. He was awarded two scholarships 
in order to support his research into and creation of detection systems, 
including authorship of one of the first Intrusion Prevention Systems that 
actually blocked penetration. Mr. Larsen has been unable to publish most of 
his work due to national security concerns. His work for the Department 
of Energy through the Idaho National Laboratories allowed him to develop 
even more elegant solutions to the security problems of major SCADA and 
PCS systems. His security work has benefited hundreds of clients among 
several industries, including US and foreign. 

I’d like to dedicate this book to the infinite patience and understanding ofThe 
Girlfriend. Thank you for the quiet nods when listening to the latest problem and 
the occasional push out the door to get some sunlight. Every geek should be required 
to have a permanent tattooed companion. 

vii 


www.allitebooks.com 


Luis Miras is an independent security researcher. He has worked for both 
security product vendors and leading consulting firms. His interests include 
vulnerability research, binary analysis, and hardware/software reversal. 

In the past, he has worked in digital design and embedded programming. 
He has presented at CanSecWest, Black Hat, CCC Congress, XCon, 
REcon, DefCon, and other conferences worldwide. When he isn’t heads 
down in IDA or a circuit board, you will likely find him boarding down 
some sweet powder. 

I dedicate this book to my parents and brothers. I would like to thank Don Omar, 
Sister Nancy, and Nasfor providing the coding soundtrack. I would like to send greetz 
to all my friends and let them know that, yes, I’m alive and no longer MIA. Thanks 
to Sebastian “topo” Muniz for the IDA discussions and bouncing ideas. 

Walter Pearce provides application security and penetration testing services 
for IOActive, and is a regular contributor to the ongoing research and 
development of advanced tools that automate IT security testing and protective 
functions. His career began at 12, and his first professional role was as the 
operator of a data center cluster for an online retailer, which led to Senior 
Programming Engineer positions at financial service firms and institutions. 
During his time in the finance industry,Walter specialized in the conception 
of internal threats and designed mitigations to reduce incidence of such 
events. Mr. Pearce is often requested by clients to provide expert application 
security services involving a variety of platforms and languages. 

To Becca, Mom, David. Love ya all. 


www.allitebooks.com 


Contents 


Chapter 1 Introduction.1 

An Overview of Code Debuggers.2 

Summary.5 

Chapter 2 Assembly and Reverse Engineering Basics.7 

Introduction.8 

Assembly and the IA-32 Processor.8 

The Stack, the Heap and Other Sections of a Binary Executable.19 

IA-32 Instruction Set Refresher and Reference.24 

Summary.35 

Chapter 3 Portable Executable and Executable 

and Linking Formats.37 

Introduction.38 

Portable Executable Format.38 

Executable and Linking Format.50 

Summary.66 

Chapter 4 Walkthroughs One and Two.67 

Introduction.68 

Following Execution Flow.68 

Reversing What the Binary Does.72 

The Processing Subroutine.74 

Solutions Fast Track.84 

Frequently Asked Questions.85 

Chapter 5 Debugging.87 

Introduction.88 

Debugging Basics.88 

Breakpoints.89 

Hardware Breakpoints.89 

Software Breakpoints.89 

Using Breakpoints.90 

Single Stepping.90 

Watches.90 

Exceptions.91 

Tracing.91 

Debugging in IDA Pro.92 

ix 


www.allitebooks.com 



































X 


Contents 


Use of Debugging while Reverse Engineering.94 

Heap and Stack Access and Modification.102 

Other Debuggers.104 

Windbg.104 

Oily dbg.105 

Immunity Debugger (Immdbg).105 

PaiMei/PyDbg.105 

GDB.106 

Summary.107 

Chapter 6 Anti-Reversing.109 

Introduction.110 

Debugging.110 

Example Overview.114 

Obfuscation.116 

Summary.136 

Chapter 7 Walkthrough Four.137 

The Protocol Problem.138 

Protocol Structure.138 

Framing and Reassembly.138 

Self Similarity.140 

Hit Marking.153 

Example Hitlist.158 

Chapter 8 Advanced Walkthrough.165 

Introduction.166 

Reversing Malware.167 

Chapter 9 IDA Scripting and Plug-ins.199 

Introduction.200 

Basics of IDA Scripting.200 

IDC Syntax.201 

Output.201 

Variables.202 

Conditionals.203 

Loops.203 

Functions.205 

Local and Global Scope.206 

Global Variables.207 

Simple Script Examples.209 

Writing IDC Scripts.213 









































Contents xi 


Problem solving with IDC.213 

The Problem.214 

Problem Background.214 

Proposed solution.216 

Possible Improvements.220 

New IDC Debugger Functionality.221 

Useful IDC Functions.222 

Reading and Writing Memory.222 

Cross References.222 

Code Xrefs.223 

Data Xrefs.224 

Data Representation.224 

Comments.225 

Code Traversal.225 

Input and Output.226 

Basics of IDA Plug-ins.227 

Module/Plug-in Resources.227 

Introducing the IDA Pro SDK.230 

SDK Layout.230 

Plug-in Syntax.231 

Setting up the Development Environment.232 

Simple Plug-in Examples.234 

The Hello World Plug-in.234 

The find memcpy Plug-in.238 

Collecting Data.251 

Displaying Data.253 

Conclusion.255 

The Indirect Call Plug-in.256 

Collecting Data.256 

User Interface.258 

Implementing the Callback.260 

dbg_bpt.260 

dbg_step_into.262 

dbg_process_exit.262 

Presenting Results.263 

Plug-in Development and Debugging Strategies.301 

Create a new IDA Development Directory.301 

Editing Configuration Files.302 

Using an Unpacked Database.302 

Enabling Exit without Saving.303 











































xii Contents 


Plug-in Arguments.303 

Scripting to Help Plug-in Development.304 

Loaders.307 

Processor Modules.308 

Third-party Scripting Plug-ins.308 

IDAPython.309 

Supported Platforms.309 

IDARub.309 

Frequently Asked Questions.310 

Index.311 













Chapter 1 


Introduction 


The theater of the information security professional has changed 
drastically in recent years. We are no longer tasked with defending 
critical organizational assets from the unwelcome inquiry of curious 
youth; we, as a community, are now faced with fending off relentless and 
technically sophisticated attacks perpetrated by organized and nation 
state-backed criminals motivated by financial or geopolitical gain. 

The prevalence of security holes in programs and protocols, the 
increasing size and complexity of the Internet, and the sensitivity of the 
information stored throughout have created a target-rich environment 
for our next-generation adversary. This criminal element is employing 
advanced polymorphic software that is specifically engineered to evade 
IDS, IPS and AV detection engines, and provide complete remote control 
and eavesdropping functionality on the victims' computer. One of the 
few offenses we can deploy in order to understand and predict the 
impact of these malicious software programs is through employment of 
advanced reverse engineering techniques, leveraging industry-standard 
tools from companies like Data Rescue and Zynamics. 

This book representsthe leading thought from the reverse engineering 
world. The authors are tremendous people in their own right, and I trust 
you and your organization will find a wealth of information that will 
help prepare you for the proactive computer security frontier. 

A big thanks to Lauren Vogt, Ted Ipsen, Dan Kaminsky, Jason Larsen, 
Walter Pearce, Justin Ferguson, Luis Miras, and the kind folks at Syngress 
for making this book possible. 

Joshua J. Pennell, Founder and CEO 
lOActive, Inc. Comprehensive Computer Security Services 
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An Overview of Code Debuggers 

Sooner or later you will want to know absolutely everything about an executable file. 
You may want to know, for instance: 

■ The exact memory address that it is calling 

■ The exact region of memory that it is writing to 

■ What region it’s reading from 

■ Which registers it’s making use of 

Debuggers will aid you in reverse-engineering a file for which you don’t have the source 
code, by disassembling the file in question. This comes in handy when you’re analyzing malware, 
as you almost never have access to the executable’s original source code. The goal of this section 
is not to coach you in depth on how to use these debuggers, but simply to show you that they 
are out there and available for you to use. Debuggers are very powerful tools that take a long 
time to learn to use to their ffillest extent. 

The “cream of the crop” in debuggers and the focus of this book is Interactive 
Disassembler Pro (IDA Pro), available from DataRescue. IDA Pro should be your first choice 
of debuggers for an enterprise environment. It isn’t really expensive, and is well worth the 
nominal outlay for the features it offers. 


DataRescue offers a demo version from their Web site at www.datarescue, 
com/idabase/index.htm. This version can only work with a limited range of file 
and processor types, is time limited, runs only as a Windows GUI application, 
and so on. 


IDA Pro is much more than a simple debugger. It is a programmable, interactive 
disassembler and debugger. With IDA Pro you can reverse-engineer just about any type 
of executable or application file in existence. IDA Pro can handle files from console 
machines such as Xbox, Playstation, Nintendo, to Macintosh computer systems, to PDA 
platforms, Windows, UNIX, and a whole lot more. Figure 1.1 shows the initial load 
screen wizard when you first start IDA Pro. Notice all the file types and tabs that will 
help you select the proper analysis for the file type that you wish to disassemble. 
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Figure 1.1 IDA Pro's Disassembly Database Chooser Loads Upon Start 



In Figure 1.2, IDA Pro has loaded and is disassembling a WootBot variant with file name 
instantmsgrs.exe. Part of what we can see from Figure 1.2 is that instanmsgrs.exe was packed 
using an executable packer called Molebox.You can also plainly see the memory calls that it’s 
making, and the Windows DLLs that are being called. This type of information can be 
invaluable when it comes to fighting off a virus or malware outbreak, especially if you need 
to make a custom cleaner in order to repair your systems. 


www.syngress.com 







4 Chapter 1 • Introduction 


Figure 1.2 IDA Pro Disassembles instantmsgrs.exe, a WootBot Variant 



Eida - C:\Documents and Settings\exibar\Desktop\instantmsgrs.exe 


Pile Ecft 3Wnp Search View Debugger Options Windows Help 


tSl IDAViewA | |^J HexV'iew-A| B Exports | &•§ Imports | N Names | i Functions] Strings) $ Structures) •• Enums | 


58 32 3? E8 4F 00 ■■■■II0LEB0X2|F0. 

80 61 i*E BH 49 71 . .egEp*"±;iaN; Iq 

A9 4B 5E 3F 80 EC aft *W|ilidJ-K*?Q8 

39 17 EE El EC 8F |•p^i?X5+19left8fi 

E3 F6 IB 3F 58 35 q pfi?X5+ipfi?X5 

00 00 E9 97 5A 00 *iTeZ..T£Z..TuZ. 

60 7C 00 00 85 B3 .FJ 

11 40 8D ID 9A A6 QIQMeJ-kXdl0ilU2 


seg005:004C1EFC ; 
seg005:004C1EFC 
seg005:004C1EFC 
seg005:004C1EFC start 
seg005:004C1EFC 
segOOS:004C1EFD 
seg005:004C1FO2 
seg005:004C1F06 


public start 


pusha 

call loc 
and byti 
sub ah. 


0001ACFC 004C1EFC; seg005:start 


lAutoanalysi s subsystem has been Initialized 


Possible file format: Portable executable for 80336 (PE) (C:\Program Files\lOA oerno 5.0\1oaders\pe.ldw) 
Loading file 'c:\Documents and setti ngs\exibar\oesktdp\instantm r — — 1 ■*-*-- — 

Detected file format: Portable executable for 80386 CPE) 
o. creating a new segment (00401000-00430932) . ok 

1. creating a new segment (0043EOOO-00442B9F).OK 

2. Creating a new segment f00443000-004BA8D8) . OK 

3. Creating a new segment (004BB000-004BCE32) . OK 

4. creating a new segment (004B0000-004C06E0) . ok 

5. Creating a new segment (004C1000-004CEC0F). ok 

6. Creating a new segment (004CF000-004CFC00) . OK 

7. creating a new segment (00400000-00407103) . OK 

Beading imports directory... 

s. creating a new segment (004CF028-004CFC00) . ok 

Assuming _cdecl calling convention by default 

Marking typical code sequences... 

File 'C:\Documents and Setti ngs\e:<ibar\Desktop\i ns tantmsgrs . exe 

rnmnilinn -Pi 1 *» 'r^Prrinram cilec'iTna n»mr> c rVS tiirN i i rlr ' 


is successfully loaded into the database, 
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Summary 

IDA is one of the most popular debugging tools for Windows. First, IDA Pro is a disassembler, 
in that it shows the assembly code of a binary (an executable or a dynamic link library [DLL]). 
It also comes with advanced features that try to make understanding the assembly code as 
easy as possible. Second, it is also a debugger, in that it allows the user to step through the 
binary file to determine the actual instructions being executed, and the sequence in which 
the execution occurs.You’ll learn about all of these features throughout this book. IDA Pro is 
widely used for malware analysis and software vulnerability research, among other purposes. 
IDA Pro can be purchased at www.datarescue.com. 
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Solutions in this chapter: 

■ Assembly and the IA-32 Processor 

■ The Stack, the Heap and Other Sections of 
a Binary Executable 

■ IA-32 Instruction Set Refresher and Reference 


0 Summary 
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8 Chapter 2 • Assembly and Reverse Engineering Basics 

Introduction 

In this chapter we will introduce basic items, providing a brief introduction to assembly and 
the Intel architecture processor and covering various other concepts that will help ease you 
into the subject matter. This book focuses on 32-bit Intel architecture (IA-32) assembly and 
deals with both the Windows and Linux operating systems. The reader is expected to be at 
least mildly familiar with IA-32 assembly (although the architecture and instruction set are 
covered to some degree in this chapter), and a firm grasp of C/C++ is expected. The point 
of this chapter is simply to give those who are either unfamiliar with or a little rusty on the 
subjects presented in this book a base to work from, and to provide a basic reference point 
to which the authors can refer should it be deemed necessary. 

Assembly and the IA-32 Processor 

Assembly is an interesting method for communicating with computers; Donald Knuth once 
said that “Science is what we understand well enough to explain to a computer. Art is 
everything else we do.” To me, this truth is most prevalent in assembly programming, because 
in so many areas as you write assembly you find yourself doing things like abusing instruc¬ 
tions by using them for something other than their intended purpose (for instance, using the 
load effective address (TEA) instruction to do something other than pointer arithmetic). But 
what is assembly exactly? Assembly refers to the use of instruction mnemonics that have a 
direct one-to-one mapping with the processor’s instruction set; that is to say, there are no 
more layers of abstraction between your code and the processor: what you write is what it 
gets (although there are some exceptions to this on some platforms where the assembler 
exports pseudo-instructions and translates them into multiple processor instructions—but the 
prior statement is generally true). 

If we take, for instance, the following single line of C code: 
return 0; 

we may end up with the following assembly being generated: 

leave 

xor eax, eax 
ret 


Note 

Many assemblers have different syntaxes. In the assembly code above, and 
indeed in assembly used throughout most of this book, the syntax employed 
is Intel syntax. Another popular syntax used largely in the Unix world is AT&T 
syntax, which looks a little bit different. The choice of Intel was made 
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because it is the syntax used by IDA in its disassemblies and is indeed a more 
popular syntax overall. This means that there will generally be more books, 
white papers and people willing to answer questions when you use the Intel 
syntax. 

The difference between AT&T and Intel syntax is outside the scope of this 
document, but just so you know what it looks like, the following example is 
given: 

Intel: 

leave 

xor eax, eax 

ret 

AT&T: 

leave 

xorl %eax, %eax 

ret 


Note, however, that AT&T is still in use in many places and is generally the 
standard syntax used in the Unix world (although this is slowly changing on 
IA-32-based computers running Unix and Unix-like OSes). Therefore, it may 
not be a bad idea to spend a few extra clock cycles at least learning it, espe¬ 
cially if you intend to do much work on the various Unices or other 
platforms. 


Don’t worry if at the moment you’re not entirely sure what that means; I’m just hoping to 
get you into the groove of assembly Just know that the previous code is IA-32 assembly, and it 
is the same as saying “return 0” in C. But this isn’t what the processor sees; assembly is the last 
layer of code that is considered human-readable. When your compiler comes through and 
compiles and assembles the code, it outputs what are known as “opcodes,” which are the binary 
representations of the instructions, or the on-off sequences necessary to execute individual 
instructions. Opcodes are typically represented in hexadecimal for humans though, since it 
tends to be easier to read than binary. The opcodes for the previous instructions are as follows: 

0xC9 (leave) 

0x31, OxcO (xor eax, eax) 

0xC9 (ret) 

As we see, these are the three basic layers of abstraction and, as advances in computing 
continue, we add more and more layers of abstraction, such as virtual machines used by Java 
and .NET applications. However, everything in the end is assembly, and that is just fixed 
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sequences of ones and zeros being sent to the processor. For a more complete discussion 
of opcodes please refer to the Intel ® 64 and IA-32 Architectures Software Developer’s Manual 
Volume 2A, section 2.1.2. 


V 


Tools & Traps... 


Opcodes and shellcode 

In almost everyone's transition into the digitally sublime, we encounter the term shell- 
code, and it strikes fear deep into our hearts. We see these character arrays of cryptic 
hexadecimal numbers and we're just not quite sure what they do. Anyone who has 
examined an exploit has probably run across it, and if you're early enough into your 
career you probably don't fully understand it. 

Rest assured, however, that these mystical thoughts about it are overcompli¬ 
cated. Shellcode is simply a series of opcodes, typically stored in a C character array. 
The term shellcode derives from the fact that the series of opcodes are the instruc¬ 
tions necessary to execute a shell, such as /bin/sh or cmd.exe. In our case, if we take 
our previous example: 

return 0; 

to generate shellcode for that particular C level instruction, we would simply use the 
opcodes for that instruction, which are 0xC9, 0x31, OxCO, 0xC9; if we put it into a C 
program it would probably look like this: 

unsigned char shellcode[] = "\xc9\x31\xc0\xc9"; 

Now that you know that, you might be inclined to feel a little silly for thinking 
it much more complicated than it is, but you shouldn't. I think everyone goes through 
that stage in their path towards enlightenment—I know I did. 


So now we have some comprehension of what assembly instructions are, but how are 
they used? An instruction that takes an argument (also known as an operand) will, depending 
on the instruction, either take a constant, a memory variable or a register. Constants are 
simple: they are statically defined in the source code. For instance, if a section of code were 
to use the following instruction: 

mov eax, 0x1234 

then the hexadecimal number 0x1234 would be the constant. Constants are fairly straight¬ 
forward and there is not much to say about them aside from the fact that they’re constant 
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and are typically encoded directly into the instruction. One interesting subject, though, is 
that if you consider our prior example of returning zero in C, the astute reader may note 
that the assembly that was generated by the compiler doesn’t contain a constant even though 
the source-level code does contain one. This is the result of an optimization performed by 
the compiler and its recognizing that copying zero is a larger instruction than performing an 
exclusive or. 

Next, we encounter registers. Registers are somewhat akin to a variable in C/C++. 

The general purpose register can contain an integer, an offset, an immediate value, a pointer 
or really anything that can be represented in 32 bits. It is basically a preallocated variable that 
physically exists in the processor and is always in scope. They are used a little differently from 
what we typically think of variables, however, as they are used and reused over and over 
again, whereas in a C or C++ program we will usually define a variable for a single purpose 
and then never use it again for something else. 

In IA-32 there are eight 32-bit general purpose registers, six 16-bit segment registers, 
one 32-bit instruction pointer register, one 32-bit status register, five control registers, three 
memory management registers, eight debug registers, and so on. In most cases you will only 
be dealing with the general purpose registers, the instruction pointer, the segment registers 
and the status register. If you’re dealing with OS drivers or similar, you’re more likely to 
encounter the other registers. Here we’re going to cover the general purpose registers, the 
instruction pointer, the status register and the segment registers. As for the others, it’s 
probably good enough just to know they exist, although naturally the interested reader 
is encouraged to consult the Intel documentation for further details. 

The eight 32-bit general purpose registers are as follows: EAX, EBX, ECX, EDX, ESI, 
EDI, EBP and ESP. These registers can mostly be used as one sees fit, with a few notable 
exceptions. For instance, many instructions assign specific registers to certain arguments (or 
operands). As a specific example, many of the string instructions use ECX as a counter, ESI 
as a source pointer and EDI as a destination pointer. Furthermore, some of these instructions 
imply the use of a given segment as a base address in certain memory models (both are 
covered shortly). Finally, other registers are implied during certain operations. For instance, 
the EBP and ESP registers are used in many stack operations and their values containing an 
address that is not mapped into the current process’s address space will often result in the 
application crashing. The IA-32 architecture is almost entirely backwards compatible to the 
8086 processor and this is reflected in their registers; all of the general purpose registers can 
be accessed in a manner yielding the register’s full 32-bit contents or its lower 16 bits, and 
the EAX, EBX, ECX and EDX registers can have their high-order and lower-order 8 bits 
accessed as well. This is accomplished by using the names reflected in Figure 2.1. For 
instance, to access the low-order 8 bits of the EAX register, you would replace EAX in your 
instruction with AT; to access the lower 16 bits of the EBP register you would replace EBP 
with BP; and to access the second set of 8 bits in the low-order 16 bits of the EDX register, 
you would replace EDX with DH. In addition to the general purpose registers, we also have 
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the instruction pointer, EIP. The EIP register points to the next instruction to be executed 
by the processor and by implication the goal of almost any application-based attack is to 
control this register. Unlike the general purpose registers, however, it cannot be directly 
modified. This is to say that you cannot execute an instruction to move a value into the 
register, but rather you would have to perform a set of operations that indirectly modify its 
value, such as a push onto the stack segment followed by a ret instruction. Don’t worry if that 
last sentence was a bit outlandish and you didn’t quite understand it yet; both of the instruc¬ 
tions referenced and the stack segment will be covered in detail later on in the chapter. Just 
for now, know that you cannot directly modify the value of the instruction pointer. 


Figure 2.1 General Purpose Registers 
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In addition to the EIP register and the general purpose register, we have six 16-bit 
segment registers: code segment (CS), data segment (DS), stack segment (SS), extra segment 
(ES), and FS and GS, which are extra general purpose segments. The segment registers contain 
a pointer to what are called segment selectors and these are often used as a base address from 
which to take an offset; for instance, consider the following instruction: 

mov DS:[eax], ebx 

In this instruction, the contents of the EBX register are copied into an offset into the 
data segment specified by EAX. Think of this as saying “the address of DS plus the contents 
of EAX.” Segment selectors are a 16-bit identifier for segments, which is to say the segment 
selector does not point directly to the segment, but rather to a segment descriptor that 
defines the segment. So, segment registers point to segment selectors, which are used to 
identify one of the 8192 possible segment descriptors that identify segments. Confused yet? 

The segment selectors are relatively simple structures. The bits at 3 through 15 are used 
as an index into one of the descriptor tables (one of the three memory management 
registers), bit 2 specifies which descriptor table exactly, and finally the low-order two bits 
specify a requested privilege level (ranging 0 through 3—privilege levels are discussed later 
in the chapter). Segment descriptors, while interesting and fairly important to OS design, are 
again not covered in order to ensure only information relevant to your general purposes is 
contained in this chapter. As always, the interested reader is encouraged to refer back to the 


www.syngress.com 









Assembly and Reverse Engineering Basics • Chapter 2 13 


Intel developer manuals. Finally, in the description of the various relevant registers, we have 
the EFLAGS register, which contains groups of various flags that indicate the states of 
previous instructions, status, and things such as the current privilege level and whether 
interrupts are enabled or not. In the current context of things, the EFLAGS register can’t 
really make sense until we have a better grasp of some of the instructions that make use of it, 
and as such a thorough description of it is reserved until later in the chapter. 

Now that we’ve described both constant and register operands, it’s time to discuss 
memory operands. Memory operands can be a little bit tricky, although at this stage in the 
game their description is pretty limited. A memory operand is by and large what a high-level 
language programmer thinks of as a variable. That is to say that when you declare a variable 
in a language like C or C++, it’s by and large going to exist in memory and thus it will 
often be a memory operand. These are typically accessed through a pointer, which when 
dereferenced will either result in the value being loaded into a register or accessed directly 
from memory. The concept itself is pretty simple, but truly understanding what is going on 
requires a bit deeper knowledge of how memory is addressed, which in turn depends on the 
memory model, mode of operation and privilege level being used. This provides an excellent 
lead into the next few paragraphs, which cover mode of operation. 

In IA-32, there are three basic operating modes and one pseudomode of operation. They 
are protected mode, real-address mode and system management mode, with the pseudomode being a 
subset of protected mode called virtual-8086 mode. In the interest of brevity, only protected 
mode will be discussed in detail. The biggest difference between the various operating modes 
is that they modify what instructions and architectural features are present. For instance, RM 
(also often called real mode) is meant for backwards compatibility, and in RM only the 
real-address mode memory model is supported. The main thing of importance to note here 
(unless you’re reversing old DOS applications or similar), is that when you reset or first 
power up an IA-32 PC, it is in real mode natively. System management mode (SMM), which 
has been in the Intel architecture since the 80386 and is used in implementing things such as 
power management and system hardware control, is also used. It basically stops all other 
operations and switches to a new address space. Generally speaking, however, almost 
everything you encounter and use will be in protected mode. 

Protected mode represented a huge advancement in the Intel architecture when it was 
introduced on the 80286 processor and further refined on the 80386 processor. One key issue 
was that previous versions of the processor supported only one mode of operation and had no 
inherent hardware-enforced protections on instructions and memory. This not only allowed 
rogue operators to do anything they wanted, but it also allowed for a faulty application to 
crash the entire system, so it was an issue of both reliability and security. Another issue with 
prior processor versions was the 640 KB barrier; this is something else that PM overcame. 
Furthermore, there were other advances, such as hardware-supported multitasking and 
modifications in how interrupts were handled. Indeed, the 286 and 386 represented 
significant advances in personal computing. 
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In the earlier days, such as with the 8086/80186 processor, or today when a modern 
processor is in real-address mode, the segment register represents the high-order 16 bits of a 
linear address, whereas in protected mode the selector is an index into one of the descriptor 
tables. Furthermore, as previously mentioned, with earlier CPUs there was no protection of 
memory or limit to instructions; in protected mode, there are four privilege levels called 
rings. These rings are given numbers for identification ranging from 0 to 3, with the lowest 
number being the highest privilege level. Ring-0 is typically used for the operating system, 
whereas ring-3 is where applications typically run.This protects the OS’s data structures 
and objects from modification by a broken or rogue application and restricts instructions 
that those applications can run (what good would the levels be if the ring-3 application 
could just switch its privilege level?). In IA-32 there are three places to find privilege level: 
in the low-order 2 bits of the CS register there is the Current Privilege Level (CPL), in 
the low-order 2 bits of a segment descriptor is the Descriptor Privilege Level (DPL), and 
in the low-order 2 bits of a segment selector is the Requestor’s Privilege Level (RPL). The 
CPL is the privilege level of the currently executing code, the DPL is the privilege level of 
the given descriptor, and, somewhat obviously, the RPL should be the privilege level of the 
code that created that segment. 

The privilege levels restrict access to more trusted components of the system’s data; for 
instance, a ring-3 application cannot access the data of a ring-2 component. However, the 
ring-2 component can access the data of a ring-3 component. This is why you can’t 
arbitrarily read data from the Windows or Linux kernel but it can read yours. Another 
function of this privilege-level separation is that it performs checks on execution transfer- 
control. A request to change execution from the current segment to another causes a check 
to be performed, ensuring that the CPL is the same as the segment’s DPL. Indirect transfers 
of execution occur via things such as call gates (which are briefly covered later). Finally, 
privilege levels restrict access to certain instructions that would fundamentally change the 
environment of the OS or operations that are typically reserved for the OS (such as reading 
or writing from a serial port). 

Moving on from protected mode, we have the three different memory models: flat, 
segmented and real-address. Real-address mode is used at boot time and is kept around for 
backwards compatibility; however, it is quite likely you will never (or very rarely) encounter it 
and thus it’s not discussed in this chapter. The flat memory model is about what you’d expect 
it to be from its name: it’s flat (see Figure 2.2)! This means basically that the systems memory 
appears as a single contiguous address space, ranging from address 0 to 4294967296 in most 
instances. This address space is referred to as the linear address space, with any individual 
address being known as a linear address. In this memory model, all of the segments—code, 
stack, data, and so forth—fall within the same address space. Despite what you may be 
inclined to first think, this is the memory model used by nearly all modern OSes and chances 
are your computer’s OS at home employs it. This seems like it would lead to disaster and 
unreliability; however, it is almost universally used with paging, which we’ll discuss shortly. 
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Figure 2.2 Flat Memory Model 
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The flat memory model in protected mode differs only in that the segment limits are set 
to ensure that only the range of addresses that actually exist can be accessed. This differs from 
other modes, where the entire address space is used and there may be gaps of unusable 
memory in the address space. 

The next memory model is the segmented memory model. It was used in earlier 
operating systems and has seen somewhat of a comeback, as in some arenas it implies an 
increase in speed (due to the ability to skip relocation operations) and security. Nonetheless, 
this memory model is still pretty rare and whether it makes a full comeback is yet to be 
seen, although it’s a bit unlikely. It’s described here because, if you do much reverse 
engineering or exploit development under Tinux, your likelihood of encountering it goes 
up considerably. 

In a segmented memory model, the systems memory is split up into subsections called 
segments. (See Figure 2.3.) These segments are kept independent and isolated from each 
other. To compare the segmented and flat memory models for a moment, what is really 
different between them is their representation to an OS or application. In both instances, the 
data is still stored in a linear fashion; however, the view of that data changes. With segmented 
memory instead of issuing a linear address to access memory, a logical address (also known as 
a far pointer) is used. A logical address is the combination of a base, which is stored in a 
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segment selector, and an offset. These two correspond to an address in that segment, which 
in turn maps to the linear address space. What this accomplishes is a higher degree of 
segmentation that is enforced by the processor, ensuring that one process does not run into 
another process (whereas in a flat memory model the same result is hopefully accomplished 
by the OS). Thus, the base address plus the offset equals a linear address in the processor’s 
address space. Furthermore, a multiple segment model can be used that retains the same traits 
as a single segmented model, except each application is given its own set of segments and 
segment descriptors. At the present time, the author cannot think of an OS that employs this, 
so a further description of how it works is moot. 

Figure 2.3 Segmented Memory Model 
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Damage and Defense. 


The Security of Segmentation 

As mentioned earlier, the segmented memory model has recently regained some 
traction in various communities, particularly in the case of grsecurity (http://grsecurity. 
com/) and PaX (http://pax.grsecurity.net/), which are third-party Linux kernel patches 
that provide superior security to that of the vanilla kernel. The lead develop of 
grsecurity. Brad Spengler, demonstrated the insecurity that a flat memory model can 
bring by product what is believed to be the first exploitable Linux kernel NULL pointer 
dereference, the details of which can be found at the following URL: http://marc.info/ 
?l=dailydave&m=117294179528847&w=2. Furthermore, the anonymous author of 
PaX has implemented a feature called UDEREF which attempts to stop accidental 
dereferences of pointers provided by user-space pointers in the kernel (and thus a 
potentially exploitable condition). This feature has been documented in regards to 
how it works and the interested reader is encouraged to read the brief write-up to 
further understand the security implications of the flat memory model which UDEREF 
fixes. At the time of this writing, it can be found at the following URL: http://grsecurity. 
net/~spender/uderef.txt 


As a result of modern OSes often using larger address space than physical memory can 
accommodate, some form of addressing all of the necessary data without requiring that it be 
stored in physical memory is required. The answer comes in the form of paging and virtual 
memory, one of the fundamental tenets of modern computing that is often misunderstood 
by people with a more operations-oriented background. It’s not uncommon to encounter 
people who understand that a given application can be given 4 GB of memory but who 
don’t really comprehend how that could possibly work, given that they only have 1 or 2 GB 
of physical memory. 

In short, paging takes advantage of the fact that only the currently necessary data needs 
to be stored in physical memory at any given moment; it stores what data it needs in physical 
memory and stores the rest on disk drive. The process of loading memory with data from 
disk, or writing data to disk, is known as swapping data, and this is why Windows boxes 
typically have swap files and Linux boxes usually have swap partitions. When paging is not 
enabled, a linear address (whether it be formed from a far pointer or not) has a one-to-one 
mapping with a physical address and no translation between it and the issued address occurs. 
However, when paging is enabled, all pointers used by an application are virtual addresses. 
(This is why two applications in a protected mode flat memory model using paging 
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accessing the exact same address don’t trample each other.) These virtual addresses do not 
have a one-to-one mapping with a physical memory address. 

When paging is employed, the processor splits the physical memory into 4 KB, 2 MB or 
4MB pages. When an address is turned into a linear address and then through the paging 
mechanism it is looked up, if the address does not currently exist in physical memory then a 
page-fault exception occurs, instructing the OS to load the given page into memory and 
then performing the instruction that generated the fault again. 

The process of translating a virtual address to a physical address varies depending on the 
page size being used, but the basic concept is the same either way. A linear address is divided 
into either two or three sections. First the Page Directory Base Register (PDBR) or Control 
Register 3 (CR3) is used to locate a Page Directory. Following this, bits 22 through 31 in 
the linear address are used as an offset into the Page Directory which identifies a given Page 
Table to use. (See Figure 2.4.) Once the Page Table has been located, the bits 12 through 21 
are used to locate a Page Table Entry (PTE), which identifies the page of memory to be 
used. Finally bits 0 through 11 of the linear address are used as an offset into the page to 
locate the data requested. When using other side pages, the process is nearly identical except 
that one layer of indirection is omitted; the directory entry points directly to the page and 
the page table and PTEs are completely omitted. The contents of Page Directory Entries 
(PDEs) and PTEs are not important for our purposes. If they become important to you at 
some point or if you’re just curious, please refer to the processor’s documentation. 


Figure 2.4 4KB Address Translation 
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Now that we have a decent understanding of instructions and operands, memory models, 
operating modes and so on, we can move on. Most of the terms employed later on in this 
chapter and throughout the book have now been defined, so you can refer back to this 
section should you feel like you don’t understand something as you work through the book. 

The Stack, the Heap and Other Sections 
of a Binary Executable 

In the previous section we talked some about segments, segment registers, segment descriptors 
and segment selectors, but we really didn’t delve into the data that they contain. 
Understanding these various sections is fairly important to understanding the layout of a 
binary executable. In this section we will discuss these concepts as much as possible, although 
in some areas, such as with the heap, it’s not really possible to jump into the depths of how it 
works exactly without a fairly in-depth analysis of a particular implementation; in those 
instances a generic high-level overview is provided and the understanding of the most 
minute details is left as an exercise for the reader. 

Warning 

The reader should understand that sections and segments as defined here do 
not implicitly or explicitly imply a segmented memory model. In all of the 
memory models, applications and OSes are split into various sections; an 
application is blissfully unaware of the implementation details. Furthermore, 
throughout this section, the terms segment and section are used 
interchangeably. 

Earlier we discussed the segment registers, in particular the CS, DS and SS segment 
registers, but we didn’t tell you what the code, data and stack segments were. In traditional 
design, an application has a few different basic sections (and a lot of implementation-specific 
ones).The basic ones employed are the code segment (or text segment or simply .text), the 
data segment (often simply .data), the block started by symbol (BSS/.bss) segment, the stack 
segment and the heap segment. As an example, the following C code will help demonstrate 
the differences between the various sections: 

unsigned int variable_zero; 
unsigned int variable_one = 0x1234; 
int 

main(void) 


void * variable_two = malloc(0x1234); 


} 
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In this code example, we have three variables defined and one function. The first 
variable, appropriately named variable_zero, is a variable of global scope that is uninitialized. 

In this instance, the C compiler will allocate space in the binary and fill it with zeros. The 
section of the binary it will exist in is the BSS section. The variable named variable_one is 
another globally scoped variable. However, in this instance it is initialized to the value 
0x1234. In this instance, the compiler will preallocate the space for the variable in the 
binary and store the value in the data segment. Following this, we have the function main. 
Main obviously is a function, and thus is in the code segment. After main we find the vari¬ 
able called variablejtwo, which gives us an interesting predicament: we have a pointer whose 
scope is inside of main and the memory that it points to. The pointer itself is local to the 
function, is dynamically allocated and exists on the stack segment, giving it a lifetime of the 
function it exists in, whereas the pointer returned by mallocf) exists on the heap, is dynamically 
allocated and has a global scope and a “use-until-free()” life expectancy. There are also often 
other sections; for instance, in programs compiled by the GNU compiler collection (GCC) a 
constant string that is declared in the source file will often end up in a section named .rodata, 
for read-only data, or in the code segment. 

The stack is one of the more important sections to understand, as it plays a vital role 
in the routine operations of an application. Those with a formal computer science 
background will no doubt know what a stack is and how it works. A stack is a simple data 
structure that basically stacks data on top of each other and has elements added and 
removed in a last-in first-out (FIFO) manner. When you add an item to the stack, you 
push it onto a stack, and when you remove an item from a stack, you pop it off the stack. 
(See Figure 2.5.) The stack is important for two basic reasons on most computing 
platforms. The first is that all automatic or local variables are stored on it; that is, when a 
function is called, any local variables it has declared that are not static or similar have their 
space allocated on the stack. This is usually realized by adding or subtracting a number 
from the current stack pointer. The stack pointer is the ESP register, and it normally 
points to the top of the stack segment, or rather SS:ESP points to the top of the stack 
segment. The top of the stack is the lowest currently in use address of the stack—the 
lowest because the stack grows down on IA-32.The bottom of the stack is usually 
bounded, not by the absolute bottom, but the bottom of the current stack frame and is 
pointed to by the EBP register. 

A stack frame is the current view into the stack relevant to the currently executing function; 
when the processor enters a new procedure, a few steps occur known as the procedure prologue. 
(See Figure 2.6.) The procedure prologue is as follows: the routine first pushes the address of the 
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Figure 2.5 A Stack 



next instruction in the calling frame onto the stack; next the current base (EBP) of the stack is 
saved onto the stack; the ESP register is then copied into the EBP register; and finally the ESP 
register is decremented to allocate space for variables in that function. When the function is 
called, the arguments to the function are pushed onto the stack in reverse order (or the last one 
is pushed first).The assembly generated for the prologue is as follows: 

push ebp 
mov ebp, esp 
sub esp, 0x1234 

Figure 2.6 Stack Frame 
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What we see here is strangely missing the saving of the return address, or the address 
we will continue execution at in the calling routine. For instance, given the following 
C code: 

A() ; 

B () ; 

when inside of the function A() the return address would be the address of the instruction 
B(). It’s a little tricky to understand at first, especially because you don’t ever see the 
instruction that saves the address onto the stack, but rather it is implied by using the call 
instruction, which is discussed a little later on in the chapter. In addition to the procedure 
prologue, there is also the procedure epilogue. The procedure epilogue basically undoes 
everything that the prologue did. This includes incrementing the stack pointer to deallocate 
any local variables, and then calling the leave and ret instructions, which remove the saved 
frame pointer and return address and return execution flow to the calling function. So, to 
summarize the points on the stack: 

■ The stack grows down, towards lower addresses on IA-32. 

■ Items are removed and added onto a stack in LIFO order. 

■ Variables that are locally scoped and only exist for the lifetime of the function end 
up on the stack. 

■ Each function has a stack frame (unless specifically omitted by the compiler) that 
contains the local variables. 

■ Prior to each function’s stack frame there is a saved frame pointer, return address 
and the parameters to the routine. 

■ The stack frame is constructed during the procedure prologue and destructed 
during the procedure epilogue. 

The heap is another important data structure, but not because any features of the 
processor depend on it, but rather because of the large amount of use it receives. The heap is 
simply a section of memory that is used for dynamically allocated variables that need to exist 
outside of the current stack frame; as a result of trait, most of the objects and indeed large 
amounts of the data an application uses will be on the heap. The heap is usually either the 
result of a random mapping or in more classic examples it was a dynamic extension of the 
data segment (although DS rarely if ever points to the heap). In that sense, the processor is 
by and large ignorant and the details are hidden away from the processor. Furthermore, the 
OS knows very little about the user-land heap; when requested, it simply gives the 
application more memory if possible and fails otherwise. It is typically the libc or similar that 
provides the heap operations and thus defines its semantics. 
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The heap, typically upon initialization, will request a fairly large section of memory from the 
OS, and will hand out smaller chunks of memory based upon requests from the application. 
These chunks will typically have inline metadata indicating the chunk’s size and other elements, 
such as the size of the previous block of memory. 

The blocks of allocated memory are navigated by taking the pointer to a given chunk 
and adding its size to it to find the next chunk, or by subtracting the previous size from the 
beginning of the chunk to find the previous one. For instance, in Figure 2.7 you will find an 
example of an allocated chunk as represented in Glibc. In this instance the pointer labeled 
mem indicates that start of data returned to the API user by malloc() or similar, whereas the 
pointer labeled chunk marks the beginning of the actual chunk. There we find that there is 
metadata including the size of the previous chunk and the size of the current chunk, along 
with flags indicating various status conditions. This chunk, while mostly used by Tinux, is 
generically similar to chunks used by most operating systems and dynamic memory allocator 
implementation (with, of course, some key differences). Since initially obtaining that large 
chunk of memory from the OS or extending the size of the data segment are fairly 
expensive operations, a cache of sorts is usually maintained. This cache usually comes in the 
form of a linked list of pointers to previously freeQ’d chunks of memory. This list is typically 
fairly complex, with the blocks of memory being coalesced into adjacent free blocks of 
memory to reduce fragmentation, and with various lists sorted by size or some other- 
characteristic to allow the most efficient means possible of locating a candidate block of 
memory whenever an allocation request occurs. 

Figure 2.7 Glibc Allocated Chunk 
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To use a similar example to the previous one, in Figure 2.8 you will see the representation 
of a free block of memory as represented by Glibc. In this instance, the pointer labeled mem 
indicates where the pointer returned to the API user used to be, and the one labeled chunk 
points to the beginning of the physical data structure used. The biggest difference is that in 
what used to be user data, there are now two pointers stored pointing to the next free block 
of memory in the linked list and the previous block of memory in the linked list. This of 
course implies that, unlike allocated chunks which are navigated by size, a free block of 


www.syngress.com 



24 Chapter 2 • Assembly and Reverse Engineering Basics 

memory is navigated directly by linked list. The specific details of the structure listed are, 
again, specific to Glibc; however, the concept itself is generic enough to apply to most 
implementations. Thus, as allocation and free requests come in and out, which happens quite 
frequently throughout the lifetime of your average application, chunks are taken away from 
the original chunk of memory obtained from the OS and returned to free lists, and then if 
possible further allocation requests make use of these blocks of memory on the free list, and 
so on until either all memory is in use, or the application terminates. 


Figure 2.8 Glibc Free Chunk 
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In the previous section, we’ve discussed the most common sections of a binary executables 
layout, including some of their functions, and took a more in-depth tour of the stack 
segment and the heap segment and talked about how they worked to some degree. This 
should be enough to provide a base to continue building your understanding. Of course, the 
interested reader is encouraged to refer to other works more specifically targeted at questions 
they may have on these subjects. 


IA-32 Instruction Set Refresher 
and Reference 

In the prior sections, we talked briefly about instructions and operands, but focused more on 
the architectural design of IA-32 and then delved into some common layouts for binary 
executable memory and their purposes and uses. In this section the intention is to provide 
you with a reference for some of the more commonly used instructions and talk to you 
some about their uses and operands. If by now you’ve already read the Intel developer 
manuals, then this section is likely to be redundant and you may wish to skip directly to the 
next chapter. In this section the terminology shown in Table 2.1 will be used. 
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Table 2.1 Terminology Employed 


Term 

Meaning 

Reg32 

Any 32-bit register 

Reg 16 

Any 16-bit register 

Reg8 

Any 8-bit register 

Mem32 

32-bit memory operand 

Mem16 

16-bit memory operand 

Mem8 

8-bit memory operand 

Sreg 

Segment register 

Memoffs8 

8-bit memory offset 

Memoffs16 

16-bit memory offset 

Memoffs32 

32-bit memory offset 

Imm8 

8-bit immediate (constant) 

Imm16 

16-bit immediate 

Imm32 

32-bit immediate 

ptr 16:16 

absolute address given in operand 

ptr 16:32 

absolute address given in operand 

mem16:16 

absolute indirect address given in mem16:16 

mem16:32 

absolute indirect address given in mem16:32 

rel8 

8-bit relative displacement 

rel 16 

16-bit relative displacement 

rel32 

32-bit relative displacement 

Register name 

Any name of a register that has already 
been introduced 


The first instruction we are going to cover is the mov instruction, which is a very basic 
instruction that copies one operand to the other. It can take the forms and allows the 
operands shown in Table 2.2. 
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Table 2.2 mov Instruction 


Destination Operand 

Source Operand 

reg8/mem8 

reg8 

reg16/mem16 

reg16 

reg32/mem32 

reg32 

reg8 

reg8/mem8 

reg16 

reg16/mem16 

reg32 

reg32/mem32 

reg16/mem16 

Sreg 

Sreg 

reg16/mem16 

AL 

memoffs8 

AX 

memoffs16 

EAX 

memoffs32 

memoffs8 

AL 

memoffs16 

AX 

memoffs32 

EAX 

reg8 

imm8 

reg16 

imm16 

reg32 

imm32 

reg8/mem8 

imm8 

reg16/mem16 

imm16 

reg32/mem32 

imm32 


The mov instruction copies the source operand to the destination operands and can only 
be used to move certain types of operands; for instance, it cannot be used to set the Code 
segment, and it cannot be used to modify the EIP register. If a destination operand is of type 
Sreg, then it must point to a valid segment selector. The next instructions introduced will be 
the various bitwise operations such as and and exclusive or. 

The and instruction is another fairly simple instruction. It performs a bitwise AND on 
the destination operand with the source operand, storing the result in the destination 
operand. It supports the operands shown in Table 2.3. A bitwise AND compares the binary 
representation of the two operands, and sets the output bit to 1 if both bits compared are 
turned on; otherwise the result is a turned-off bit. 
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Table 2.3 and Instruction 


Destination Operand 

Source Operand 

reg8/mem8 

reg8 

reg16/mem16 

reg16 

reg32/mem32 

reg32 

reg8 

reg8/mem8 

reg16 

reg16/mem16 

reg32 

reg32/mem32 

AL 

imm8 

AX 

imm16 

EAX 

imm32 

reg8 

imm8 

reg16 

imm16 

reg32 

imm32 

reg8/mem8 

imm8 

reg16/mem16 

imm16 

reg32/mem32 

imm32 

The next instruction referenced is the not instruction, another fairly simple but 
commonly used instruction; it performs a bitwise NOT operation on a single operand and 

allows the operands shown in 
operand. 

Table 2.4. This simply sets each 1 to 0 and vice versa in its 

Table 2.4 not Instruction 


Destination Operand 

Source Operand 

reg8/mem8 

N/A 

reg16/mem16 

N/A 

reg32/mem32 

N/A 


Chugging forward, we have the or instruction, which performs a bitwise OR on its 
arguments and takes the arguments shown in Table 2.5. A bitwise OR is, again, a fairly 
simple operation that’s used often. It compares the two operands bit by bit and sets the 
corresponding output bit to zero only if both compared bits are set to zero. 
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Table 2.5 or Instruction 


Destination Operand 

Source Operand 

reg8/mem8 

reg8 

reg16/mem16 

reg16 

reg32/mem32 

reg32 

reg8 

reg8/mem8 

reg16 

reg16/mem16 

reg32 

reg32/mem32 

AL 

imm8 

AX 

imm16 

EAX 

imm32 

reg8 

imm8 

reg16 

imm16 

reg32 

imm32 

reg8/mem8 

imm8 

reg16/mem16 

imm16 

reg32/mem32 

imm32 

Next in the line-up we have the exclusive-or instruction, or xor. It performs a bitwise 
exclusive-or (XOR) on its operands and takes the operands shown in Table 2.6. The xor 
instruction compares the source and the destination operands and stores the output in the 

destination operand. Each output bit is 
the output bit is 0. 

1 if the two compared bits are different; otherwise 

Table 2.6 xor Instruction 


Destination Operand 

Source Operand 

reg8/mem8 

reg8 

reg16/mem16 

reg16 

reg32/mem32 

reg32 

reg8 

reg8/mem8 

reg16 

reg16/mem16 

reg32 

reg32/mem32 

AL 

imm8 


Continued 
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Table 2.6 Continued 


Destination Operand 

Source Operand 

AX 

imm16 

EAX 

imm32 

reg8 

imm8 

reg16 

imm16 

reg32 

imm32 

reg8/mem8 

imm8 

reg16/mem16 

imm16 

reg32/mem32 

imm32 


The test instruction is commonly used to determine a specific condition and then 
modify control flow based on the results (see Table 2.7).The test instruction performs a 
bitwise AND of the first and second operands, and then sets flags in the EFTAGS register 
accordingly. Following this, the result is then discarded. 

The cmp instruction compares two operands; this comparison is performed by subtracting 
the source operands from the destination operands and setting flags in the EFLAGs register 

Table 2.7 test Instruction 

Destination Operand 

Source Operand 

reg8/mem8 

reg8 

reg16/mem16 

reg16 

reg32/mem32 

reg32 

AL 

imm8 

AX 

imm16 

EAX 

imm32 

reg8 

imm8 

reg16 

imm16 

reg32 

imm32 

reg8/mem8 

imm8 

reg16/mem16 

imm16 

reg32/mem32 

imm32 
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accordingly (see Table 2.8). It is often used in a manner similar to the test instruction and is 
used to compare values like user input and return values from routines. If an immediate 
value is used as an operand, it is sign-extended to match the size of the other operand. 


Table 2.8 cmp Instruction 


Destination Operand 

Source Operand 

reg8/mem8 

reg8 

reg16/mem16 

reg16 

reg32/mem32 

reg32 

reg8 

reg8/mem8 

reg16 

reg16/mem16 

reg32 

reg32/mem32 

AL 

imm8 

AX 

imm16 

EAX 

imm32 

reg8 

imm8 

reg16 

imm16 

reg32 

imm32 

reg8/mem8 

imm8 

reg16/mem16 

imm16 

reg32/mem32 

imm32 


The load effective address instruction, or lea, calculates the address as specified by the source 
operand and stores it in the destination operand (see Table 2.9). It is also used as a method 
for doing arithmetic between multiple registers without modifying the source operands. 

Table 2.9 lea Instruction 

Destination Operand 

Source Operand 

reg8 

mem8 

reg16 

mem16 

reg32 

mem32 
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The jmp instruction transfers execution control to its operand. This instruction can 
execute four different types of jumps: a near jump, a short jump, a far jump and a task switch 
(see Table 2.10). A near jump is a jump that occurs within the current code segment. A short 
jump is a jump to an address within —128 to 127 from the current address. A far jump can 
take control to any segment in the address space providing it is of the same privilege level as 
the current code segment. Finally, a task switch jump is a jump to an instruction in a 
different task. 


Table 2.10 jmp Instruction 


Destination Operand 

Source Operand 

rel8 

N/A 

rel 16 

N/A 

re 132 

N/A 

reg16/mem16 

N/A 

reg32/mem32 

N/A 

ptrl 6:16 

N/A 

ptrl 6:32 

N/A 

mem16:16 

N/A 

mem16:32 

N/A 

The jcc instructions are not one particular instruction, but rather a series of conditional 
jumps. The conditions vary with the instruction used, but they’re typically used in 
collaboration with the test and cmp instructions. Table 2.11 shows destination operands for jcc 

instructions. In Table 2.12 you 

will find a list of conditional jumps and the flags that they 

check to determine whether a 

condition is true or not. Don’t worry about the flags just yet; 

the description of the EFLAGS register will come just after the instructions. 

Table 2.11 jcc Instructions 


Destination Operand 

Source Operand 

rel8 

N/A 

rel 16 

N/A 

re 132 

N/A 
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Table 2.12 Conditional Jump Registers 


Instruction 

EFLAGS condition 

Description 

J a 

CF = 0 && ZF = 0 

Jump if above 

jae 

o 

ii 

b 

Jump if above or equal 

jb 

CF= 7 

Jump if below 

jbe 

Q 

II 

—A 

II 

—A 

Jump if below or equal 

jc 

CF= 1 

Jump if carry 

jcxz 

cx = o 

Jump if CX is zero 

jecxz 

ECX = 0 

Jump is ECX is zero 

je 

ZF = 7 

Jump if equal 

jg 

ZF = 0 <§<§ SF = OF 

Jump if greater than 

jge 

SF= OF 

Jump if greater than or equal to 

jl 

SF != OF 

Jump if less than 

jle 

ZF = 1 || SF != OF 

Jump if less than or equal to 

jna 

3 

II 

—A 

II 

—A 

Jump if not above 

jnae 

CF= 1 

Jump if not above or equal 

jnb 

o 

ii 

b 

Jump if not below 

jnbe 

CF = 0 && ZF = 0 

Jump if not below or equal 

jnc 

o 

ii 

b 

Jump if not carry 

jne 

ii 

o 

Jump not equal 

jng 

ZF = 1 || SF != OF 

Jump not greater 

jnge 

SF != OF 

Jump not greater or equal 

jnl 

SF = OF 

Jump not less 

jnle 

ZF = 0 && SF = OF 

Jump not less or equal 

jno 

OF = 0 

Jump if not overflow 

jnp 

o 

ii 

b 

Jump not parity 

jns 

SF = 0 

Jump not signed 

jnz 

ii 

o 

Jump not zero 

jo 

OF = 1 

Jump if overflow 

P 

PF= 1 

Jump if parity 

jpe 

PF= 1 

Jump if parity even 

jpo 

o 

ii 

b 

Jump if parity odd 

js 

SF= 1 

Jump if signed 

jz 

ZF = 7 

Jump if zero 


www.syngress.com 






Assembly and Reverse Engineering Basics • Chapter 2 33 


So, as you can see in Table 2.12, there are quite a few conditional jump registers, and all 
of them depend on the various states of the EFLAGS register, which we haven’t really 
described yet, but will do now. The EFLAGS register is a 32-bit register that contains a 
group of status and system flags, and a control flag. Each flag is represented by a single bit in 
this register, and moving from bit 0 to 31 we have the following flags: 

CF Carry flag, indicates a carry or a borrow out of the most significant bit of a 
register in an arithmetic operation. This flag is used in unsigned arithmetic. 

PF Parity flag, set if the least significant bit of the result contains an even number of 
bits turned on. 

AF Adjust flag, set if an operation resulted in a carry or borrow from bit 3. 

ZF Zero flag, set if the result of an operation is zero. 

SF Sign flag, set to the most significant bit of a result (which is the sign bit in a 
signed integer) 

TF Trap flag, set to enable single-stepping debugging or when single-stepping 
debugging is enabled. 

IF Interrupt enable flag, set when maskable interrupts are enabled, cleared when they 
are blocked. 

DF Direction flag, used in string operations to determine whether the instructions 
increment or decrement 

OF Set if the result of an operation resulted in an integer overflow when performing 
signed arithmetic. 

IOPL flags (bits 12 and 13) 1/ O privilege level flags, indicates the current 1/ O 

privilege level of the currently running task 

NF Nested task flag, set when the current task is associated with the previously 
executed task 

RF Resume flag, controls the processors response to debug exceptions 
VM Virtual-8086 flag, set to enable virtual 8086 mode 

AC Alignment check flag, set to enable alignment checking of memory references 

VIF Virtual interrupt flag, virtual image of the IF flag, used together with the VIP flag 

VIP Virtual interrupt flag, used to determine if an interrupt is pending 

ID Identification flag, used to determine if the CPU supports the CPUID 
instruction. 

Bits 22 through 31 are currently reserved. 
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The call instruction is somewhat akin to a more official jump; it sets up the stack as 
previously described in a manner that will allow the processor to resume execution in the 
calling function when the executed function finishes. (See Table 2.13.) 


Table 2.13 call Instruction 


Destination Operand 

Source Operand 

rel 16 

N/A 

re 132 

N/A 

reg16/mem16 

N/A 

reg32/mem32 

N/A 

ptrl 6:16 

N/A 

ptrl 6:32 

N/A 

mem16:16 

N/A 

mem16:32 

N/A 

The ret instruction is the inverse 

of the call instruction. It takes the metadata stored on 

the stack, pops it off and returns to that address (see Table 2.14). The optional immediate 
operand specifies how many bytes to pop off of the stack after performing the return. 

Table 2.14 ret Instruction 


Destination Operand 

Source Operand 

N/A 

N/A 

imm16 

N/A 


As you can see, there are already a lot of instructions to be familiar with, and most of 
them take many different operands, which in turn results in many different forms of the 
same instruction (and thus different opcodes). This is hardly a complete instruction 
reference—in fact, it barely scratches the surface. However, we wanted to touch base on 
some of the more commonly used instructions and at least familiarize you with them. 
Readers are strongly encouraged to consult the Intel developer manuals, specifically 3A and 
3B, if they are not already familiar with the instruction set. 
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Summary 

So now you’re at least familiar with assembly and the Intel architecture. You know how the 
memory models and operating modes work to some degree and have a decent general base 
of comprehension to build on. However, you might find yourself thinking, “Yes, I understand 
some assembly now, but what is reverse engineering?”. Well, reverse engineering is a broad 
term and means different things to different people. In general, the answer is that it’s the 
process of taking an application that is in a form not meant for human readability or analysis 
and working backwards towards the beginning. Some people do this in order to regain lost 
source code, others do it to duplicate proprietary products, some reverse malicious applica¬ 
tions to know what they do, and finally others reverse engineer software to find bugs and 
vulnerabilities in it. 

To be brief, and for the purposes of this book, reverse engineering is taking a binary file 
that is meant to be read by the computer and using the opcodes to generate assembly, and 
then reading that assembly to help accomplish whatever goals you may have. In this sense, it 
can be said that, to the reverse engineer, there is no such thing as closed source software. The 
processor sees every instruction, and so does the reverse engineer. 
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Introduction 

In this chapter we will introduce two common binary executable formats, Portable 
Executable (PE) and Executable and Linkable Format (ELF). PE is the binary format used in 
Windows, while ELF is used by many of the Unices. ELF is a replacement for the older 
a.out format that did not include standardized support for shared libraries. Furthermore, 

PE is an offshoot of the COFF format, which was used in an earlier Unix, and the authoris 
understanding is that this is by and large the result of many of the developers being hired 
at Microsoft. 

At any rate, this chapter serves as an introduction to the physical layout of the Mes, and 
details aspects of the Mes that a reverse engineer would Mnd interesting and/or important. 
Both of these Me formats have open documentation and, in places where readers Ynd this 
chapter lacking, they are strongly encouraged to read the speciYcations themselves. 

Portable Executable Format 

Portable Executable format, more correctly termed the Portable Executable and Common 
Object File Format (PE-COFF), is a fairly simple format that is easily understood. Here we 
will cover the absolute basics and try to avoid covering topics that were deMned in the 
previous section (for instance, little attention is given to the .text or code section, or .data 
section). Instead weill focus on getting the reader up to speed on being able to open a Me in 
a hex editor and navigate to the various headers of the Me. With the exception of a few 
important sections, the internal format is not deYned, with this left as an exercise for the 
reader. 

In Figure 3.1 you will Ynd a basic diagram dictating the general layout of a typical PE 
Me. At the beginning you will Mnd a DOS stub program, with a header. This is a simple pro¬ 
gram designed to be run if the application is run in DOS; it is used for backwards compati¬ 
bility. To the reverse engineer, the only important part of this is that it contains an offset to 
the PE header. This can be found by simply seeking to the offset 0x3C from the beginning 
of the Me. This offset in turn dictates an offset relative to the beginning of the Me where the 
PE header can be found. 

It should be noted that the DOS header itself is only found in image Mes and not in 
other Mes such as objects and so on. One characteristic of the DOS Me header is that the 
Mrst few bytes will contain the imagici bytes indicating that it is a DOS image, speciM cally 
the characters M and Z.This knowledge, combined with other simple heuristics, can help a 
reverser identify a Me as being a PE. Skipping past the DOS header information, as it is 
essentially useless to us, we Mnd the PE header itself. The PE header begins with another 
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Figure 3.1 Typical PE Layout 



Yeld that identiYes it as a PE Yle, speciYcally the bytes PE\0\0, where \0 is a binary zero. 
Immediately following this signature, there is the COFF Yle header (see Figure 3.2).The 
Yrst Yeld in the format is the machine type; it is two bytes long and will almost always be 
either 0x8664 (AMD64), 0x14c (IA32) or less often 0x200 for IA64 Itanium processors. 
Immediately following this Yeld is one that indicates the number of sections in the Yle. It is 
also a two-byte Yeld, and, according to Microsoft documentation, the maximum value this 
can hold is 96. Skipping along to the next item of interest, we have a Yle offset to the 
COFF symbol table. As a reverse engineer, you can almost guarantee that the Yle youire 
inspecting will not have symbols, so you can expect that this Yeld will be zero, indicating 
that an offset does not exist. However, if an offset does exist, it will be speciYed here. After 
this Yeld, there is another four-byte Yeld indicating the number of symbols present. Finally, 
the last two standard COFF Yelds are both two bytes, one indicating the size of the 
optional header (which exists in images only) and a Yeld called characteristics that deYnes 
speciYc attributes of the Yle. 
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Figure 3.2 COFF File Header 


Offset 

Size 

Field Name 

O 

2 

Machine 

2 

2 

Number Of Sections 

4 

4 

Time Date Stamp 

8 

4 

Pointer To Symbol Table 

12 

4 

Number Of Symbols 

16 

2 

Size Of Optional Header 

18 

4 

Characteristics 


Bold indicates discussed/referenced field 


The characteristics specify various attributes of the Me, such as whether the Me is a 
dynamically loaded library (DLL) or not, if the Me is part of the system, if the Me has had its 
relocation information stripped, if the Me uses 32-bit words, and so on. For a full table 
describing this information, please consult the ofYcial Microsoft documentation. 

The optional header, if one exists (it does not exist in object Mes), is broken into three 
major sections, the Yrst being eight Yelds that are generic to the COFF format, followed 
by 21 Windows-speciYc Yelds, and Ynally the data directories. (See Figure 3.3.) In the 
generic COFF section, the Yelds that we would Ynd most interesting are the magic, size of 
code, size of initialized data, size of uninitialized data, entry point and code base address, and data 
base address Yelds. The magic Yeld is another signature that can be used to identify PE 
versions; the valid Yelds are OxlOB and 0x20B for PE32 and PE32+, respectively.This 
chapter will cover only the PE32 format, as it is more common, and will leave the PE32+ 
format as yet another exercise for the reader. The size Yelds are relatively self-explanatory 
and indicate the size of the .text/code, .data and .bss sections of the Me. Finally, we have 
the relative virtual address (RVA) of the entry point, or place where the application should 
start executing. An RVA is just a complicated way of referring to an offset from the virtual 
address (VA) where the Me is loaded, and should be looked at as such. Finally, after all of 
the above, we have the base virtual addresses of the code and data sections. Next, in the 
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Windows-speciYc section of the optional header, we Ynd the following Yelds of interest: 
image base, size of image, size of headers, DLL characteristics, and Ynally the number of 
data-directory entries and their size. 


Figure 3.3 Optional Header Layout 


Offset 

Size 

Optional Header Section 

0 

2 

Standard COFF fields 

28 

68 

Windows Specific fields 

96 

Variable 

Data directories 


Bold indicates discussed/referenced field 


The image base speciYes the preferred address of the Yrst byte of the image when 
loaded. According to Microsoft documentation, it must be a multiple of 64 K. The default 
varies by operating system and is not really important; itis merely a preference thing and the 
loader has the option of overriding what the Ye thinks is right. Next, there is the size of the 
image and the size of the headers; the size of the image is the total size of the Yle including 
all of the headers as loaded into memory, with the size of the headers specifying the size of 
the headers, obviously. The DLL characteristics Yeld is obviously used only for DLLs and 
speciYes attributes speciYc to the DLL. To the reverser, the interesting Yelds are 0x0040, 
which speciYes that the base address can be assigned dynamically and would allow for things 
such as address space layout randomization (ASLR), 0x0080 which speciYes that code integ¬ 
rity checks are made, 0x0100 which speciYes that the image is no-execute (NX) compatible, 
and Ynally 0x0400 which indicates that the Yle does not use structured exception handling 
(SEH), effectively preventing any SEH handler from pointing into this DLL. Finally, the last 
element of interest speciYes the number of elements in the next subsection of the optional 
header. See Figure 3.4. 
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Figure 3.4 Optional Header 
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Data directories are a bit of a different beast. They specify some other type of data for the 
image. For instance, the import table data directory speciYes which libraries and functions 
will be imported by the application for use. The data directory section of the optional header 
is an array of 16 structures containing double word values, specifying the virtual address and 
size for a given data directory. The data directories speciYed are shown in Table 3.1. 


Table 3.1 Data Directories 


Name 

Description 

Export table 

Export table specifies functions 
exported by the file 

Import table 

Import table specifies functions 
imported by the file 

Resource table 

Resource table specifies various 
resources used by the file, such as icons 

Exception table 

Exception table specifies registered 
exception handlers used by the file 

Certificate table 

Attribute certificate table 

Base relocation table 

Base relocation table specifies all base 
relocations in the file 

Debug 

Used for storing compiler-generated 
debugging information 

Architecture 

Reserved, must be zero 

Global pointer 

RVA of the value to be stored in the 
global pointer register 

Thread local storage (TLS) table 

Specifies information used in thread- 
specific data storage 

Load configuration table 

Different uses for different Windows 
versions, since XP used to register 
SafeSEH functions 

Bound import 

Bound import table 

Import address table (IAT) 

Prior to runtime, identical import 
lookup table, at runtime filled with 
resolved symbol addresses 

Delay import descriptor 

Similar to the import table but delays 
imports 

CLR runtime header 

CLR runtime header 

Reserved 

Reserved, must be zero 
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At this point, only the export, import and load conYguration tables are described in 
detail. Everything else is left as an exercise for the reader. The export table, as previously 
indicated, speciYes the functions exported by the Yle. The format of the export table, also 
known as the .edata section, is shown in Table 3.2. 


Table 3.2 Format of the Export Table 


Name 

Description 

Export directory table 

The export directory table describes the 
entirety of the export information. 

It contains address information that is 
used to resolve imports to the exported 
functions within the image. 

Export address table 

Contains the address of exported entry 
points, data and absolutes 

Name pointer table 

Array of RVAs into the export name table 

Ordinal table 

Array of 16-bit ordinals into the export 
address table 

Export name table 

Null-terminated variable length string 
names of exported functions/data/etc. 


It should be noted that not all of these tables are required to be present; if exports are only 
to be done via ordinal, then only the export directory table and export address table are 
required. The interesting Yelds of the export directory table (EDT) are: the name RVA, ordinal 
base, address table entries, number of name pointers, export address table (EAT) RVA, name pointer 
RVA, and ordinal table RVA. See Figure 3.5. 

The name RVA is the RVA to the name of the DTE in question. The ordinal base is 
simply the base index that ordinal indexing starts from; this is typically set to one. The address 
table entries and number of name pointers Yelds specify how many entries there are in the 
address table and name table, respectively. The EAT RIA, name pointer RVA and ordinal table 
RVA entries are all exactly what they sound like; they indicate the RVA for the rest of the 
tables in the .edata section. The export address table is a fairly simple structure with only one 
element that can be represented one of two ways, and is most likely implemented as a union. 
If the address is not within the export section (which is deYned by summing the address and 
length as provided in the optional header), the Yeld is an actual address in the code or data. 
Otherwise the Yeld is a forwarder RVA, which names a symbol in another DLT. The export 
name pointer table is another simple structure; it simply contains an RVA into the export name 
table for each export, if deYned. An export name is only deYned if a pointer is contained in 


www.syngress.com 


Portable Executable and Executable and Linking Formats • Chapter 3 


Figure 3.5 EDT 


Offset Size 


Reid Name 

Export Flags 


Time/Date Stamp 

Major Version 


Minor Version 


Name RVA 

Ordinal Base 

Address Table Entries 

Number Of Name Pointers 

Export Address Table RVA 

Name Pointer RVA 


Ordinal Table RVA 


Bold indicates discussed/referenced field 


this table. The ordinal table is an array of 16-bit indexes biased by the ordinal base into the 
EAT. The ordinal table and the export name table are essentially mirrors of each other in 
that an index into one provides an index into the other, providing of course that the name 
for the export exists. See Figure 3.6. 


Figure 3.6 EAT 



Offset 
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Feld Name 


Export RVA 


Forwarder RVA 
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Finally, the export name table contains the actual string that makes up the public name for 
the exported symbol; public means that if one exists, an application can import the function/ 
data by name. So, taking all of this into account, a symbol can be resolved by name using the 
following steps: 

0. Obtain theVA or the export directory table in the optional header. 

1. Use thatVA to locate the ordinal base, export directory table and the ordinal table 
RVAs. 

2. Retrieve the RVA of the name pointer RVA. 

3. Search the export name pointer table to determine if the function is exported by 
name. 

4. Use the index into the name pointer table as an index into the ordinal table to 
retrieve the ordinal. 

5. Take the ordinal and subtract it from the ordinal base and use the result as an index 
into the EAT. 

6. The data at this index is the RVA for the exported function. 

The process for obtaining a symbol via ordinal is exactly the same. However, the steps 
for Ynding the export by name are removed and the conversion from name pointer index to 
ordinal index is also removed. 

The import table, or .idata section, uses a method similar to the export table, 
although itis a little less convoluted. There are three main structures used when importing 
a symbol: the import directory table (IDT as shown in Figure 3.7), import lookup table 
(ILT as shown in Figure 3.8), and the hint/name table. The IDT contains a few Yelds; the 
ones discussed here are the ILT RVA, the name RVA, and the IAT RVA. All of these are 
fairly self-explanatory except for the name RVA, which is also simple enough in that it is 
the RVA of the null-terminated ASCII string of the name of the DLL to be imported. 
The ILT and IAT are arrays of 32-bit integers (on PE32), with each entry being a 
bit-Yeld.The high-order bit of an entry indicates whether the import is done by name or 
by ordinal; if the bit is set it is imported by ordinal. If the import is done by ordinal, bits 
0 to 15 represent the ordinal to import. If itis being imported by name, then bits 0 to 30 
represent a 31-bit RVA into the hints/name table for the name of the import. The hints/ 
name table is yet another fairly simple table. The Yrst two bytes of each entry serve as a 
ihinti to the loader. 


www.syngress.com 


Portable Executable and Executable and Linking Formats • Chapter 3 47 


Figure 3.7 IDT 
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Import Lookup Table RVA 

4 

4 

Time/Date Stamp 

8 

4 

Forwarder Chain 

12 

4 

Name RVA 

16 

4 

Import Address Table RVA 


Bold indicates discussed/referenced field 


Figure 3.8 ILT 


Bits 

Size 

Reid Name 

31 

1 

Ordinal/Name Flag 

15-0 

16 

Ordinal Number 

30-0 

31 

Hint/Name Table RVA 


Bold indicates discussed/referenced field 
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This hint is used as an index into the export name pointer table in the target DLL. If the 
entry matches, then this is used; otherwise a search for the name is performed. The next 
element is a variable length null-terminated ASCII string that is the name of the function to 
import, potentially followed by a trailing null padding byte, in order to have the next entry 
properly aligned. 


Note 

An interesting side note is that, while the IAT and ILT are supposed to 
contain the same data until the symbols are actually bound, the author has 
found that this was not always the case. In the distant past, while writing a 
tool to parse the PE format, it was found that some compilers would move 
the ILT into the .text segment and its contents would actually be different 
from expected! It was also found that some compilers didn't make use of the 
ILT at all and instead only used the IAT. When manually parsing the format, 
be aware of subtle nuances like this. Both Microsoft and Borland like to take 
shortcuts when possible. 


Finally, we move into the Load Configuration structure, as shown in Figure 3.9.This 
structure was supposedly used in limited cases in Windows NT in a very different manner 
from how it is used in the post-Windows 2000 world. In Windows XP and later, this 
section is used by SafeSEH to register valid exception handlers with the system, thus 
avoiding the issue of an attacker overwriting an SEH entry and causing an exception to be 
raised and thus having their code executed. If the IMAGE_DLLCHARACTERISTICS_ 
NO_SEH Yeld is not set in the DLL Characteristics Yeld of the optional header, and an 
exception handler is not in this list when the system is attempting to call it, then the 
process is aborted. In the Load Configuration structure, there are only three Yelds we would 
Ynd interesting: the security cookie, structured exception (SE) handler table, and structured 
exception handler count. The security cookie is not actually the cookie itself, but rather a 
pointer to it. This cookie is used in a number of ways, most notoriously when the /GS 
Ehgs are speciYed to the Microsoft compiler, which implements stack cookies to prevent 
stack-based buffer overEbws.The SE handler table is a sorted table of RVAs that correspond 
to valid SEH handlers for that particular image. The SE handler count is a count of the total 
number of handlers. 
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Executable and Linking Format 

The Executable and Linking Format (ELF) was the result of work done at Unix System 
Laboratories and was eventually published as part of the System V Application Binary 
Interface (ABI) and then later adopted in Tool Interface Standard. Interestingly enough, the 
original name for the format was Extensible Linking Format, most likely a result of many 
prior Me formats not supporting the dynamic linking of external libraries. Since its ofMcial 
adoption as the Me format of choice for Unix and Unix-like operating systems, it has 
become the de facto standard across the board in the Unix world, with nearly every vendor 
either using it as their native format or supporting it through a thin abstraction layer. It is 
used in everything from Linux, Solaris, IRIX and the BSDs to the Playstation. Therefore, 
unless the world in which you operate is entirely Windows based, you will encounter ELF 
Mes in pretty short order. 

In the ELF header (shown in Figure 3.10), there are no entirely strict sizes; everything 
is deMned relative to the native sizes of the processor, and a lot of processors use it. For this 
reason, this chapter only references IA32.The ELF header and indeed the format are a lot 
more straightforward and interesting to us as reverse engineers, as you might ascertain from 
the number of elements in the header that are touched upon in this chapter. The e_ident 
Meld identiMes the ELF Me: it identiMes the Me as an ELF Me, identiMes the native word 
size of the processor, speciMes the intended byte-ordering, and Mnally the version of the 
ELF header. The Meld is an array of unsigned characters, 16 in total. The Mrst four bytes are 
the imagici M eld with the values of 0x7F, E, L and F. The next byte speciMes the word size 
or class, with a value of 0 indicating that it is an invalid class, a value of 1 indicating that it 
is 32-bit, and a value of 2 indicating that it is 64-bit. The next byte speciMes the encoding 
of data within the Me, with a possible value of 0, 1 or 2. Zero again signiMes an invalid 
value, 1 indicates that the data is encoded in twois complement values with the least 
signiMcant byte occupying the lowest address, and 2 indicating that the encoded values are 
encoded in twois complement values with the most signiMcant byte at the lowest address. 
The next byte in the e_ident Meld speciMes the version of the ELF header and, as we will 
see shortly, is somewhat redundant. It should be set to 1, indicating that it is using the cur¬ 
rent version of the ELF speciMcation. Finally, the rest of the bytes in the array are currently 
unused and reserved. They currently serve as padding and the speciMcation suggests that 
programs parsing the header ignore any values in the Meld. Following the e_ident Meld is the 
e_type Meld, which identiMes what type of executable the Me is. The possible values are 0 
indicating that there is no Me type, 1 indicating that the Me is a relocatable Me, 2 indicating 
that it is an executable Me, 3 that it is a shared object Me, and OxFFOO and OxFFFF are 
marked as being processor speciMc. 
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Figure 3.10 ELF Header 


Offset 

Size 

Field Name 

0 

16 

eident 

16 

2 

e type 

18 

4 

e machine 

22 

4 

eversion 

26 

4 

e entry 

30 

4 

ephoff 

34 

4 

e shoff 

38 

4 

e flags 

42 

2 

e ehsize 

44 

2 

e phentsize 

46 

2 

e phnum 

48 

2 

eshentsize 

50 

2 

eshnum 

52 

2 

e shstrndx 


Bold indicates discussed/referenced field 


After the e_type Yeld, we have the e_machine Yeld, which indicates the type of processor 
the Yle was built for. The only value of substance for us is the value 3, which indicates that it 
is for an IA32 machine; the other values are for more esoteric architectures, such as SPARC, 
Motorola, MIPS and IA 8086. e_version has the same values as the version in the evident 
header, with 0 indicating that the version is invalid and 1 indicating that the version is the 
current version. The e_entry member sounds exactly like what it isoit holds theVA of the 
entry point of the application if applicable; otherwise the Yeld is set to O.The e_phoff Yeld 
yields the Yle offset in bytes of the Yleis program header table. The e_shqff Yeld yields the Yle 
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offset in bytes of the Yleis section header table; both of these Yelds e_phoff and e_shoff are 
only present if the Me has the table. Otherwise theyire initialized to 0. The e_flags Yeld 
contains processor-speciYc Digs. However, the IA32 architecture speciYes no Digs and there¬ 
fore the Yeld will be (should be) O.The e_ehsize Yeld holds the size of the ELF header, the 
e_phentsize and e_shentsize indicate the size in bytes of one entry in the program header table 
and section header table, respectively The e_phnum and e_shnum Yelds indicate the number 
of entries in the program header table and the section header table. Thus, to calculate the 
size of the program header table, you would multiply the e_phentsize and the e_phnum Yelds. 
Once again, if any of these tables do not exist, the Yeldis values are initialized to 0. Finally, at 
the end of the ELF header we have the e_shstrndx Yeld, which holds the index for the 
section name string table in the section header table, if applicable. 

The ELF section header table is an array of section header structures. Each structure is of 
the format displayed in Figure 3.9 and is again fairly straightforward. That said, in the array 
there are certain indexes that hold special values; these special indexes are shown in Table 3.3. 


Table 3.3 Special Indexes for ELF Section Header Table Array 


Name 

Index/Value 

SHNJJNDEF 

0 

SHN_LORESERVE 

OxFFOO 

SHN_LOPROC 

OxFFOO 

SHN_HIPROC 

OxFFIF 

SHN_ABS 

OxFFFI 

SHN_COMMON 

0xFFF2 

SHNJHIRESERVE 

OxFFFF 


SHN_UNDEF marks an undeYned, absent or generally meaningless section reference. 

It is important to note that, although undeYned, the section header table (if present) always 
contains an SHN_UNDEF entry at index zero; thus, if the e_shnum Yeld states that there are 
ten Yelds, there are actually nine plus the SHN_UNDEF entry. SHN_LORESERVE speci¬ 
Yes the lower bound of reserved index ranges. SHN_LOPROC and SHN_HIPROC specify 
the range of entries reserved for processor-speciYc entries. SHN_ABS speciYes absolute 
values for the relevant symbols. This essentially means symbols referenced are not affected by 
relocation. SHN_COMMON is for common symbols such as unallocated external variables 
in C, and Ynally SHN_H I RESERVE speciYes the upper bound of reserved index ranges. 
Every section in an ELF Yle has exactly one section header describing it; the sections 
described are contiguous although potentially empty and the sections themselves may not 
overlap. The elements of a section header are as listed in Figure 3.11. 
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Figure 3.11 Section Header 


Offset 

Size 

Field Name 
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2 

sh name 

2 

2 

sh type 

4 

2 

sh flags 

6 
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sh addr 

10 

4 

sh offset 

14 

2 

sh size 

16 

2 

sh link 

18 

2 

shinfo 

20 

2 

shaddralign 

22 

2 

sh entsize 


Bold indicates discussed/referenced field 


The sh_name element is an index into the section header string table, which speciYes the 
name of the section. The shjtype element determines the type of section contents and 
semantics; speciYcally the following types are deYned in Table 3.4. 


Table 3.4 Types of Section Contents and Semantics 


Name 

Value 


SHTJMULL 

0 


SHT_PROGBITS 

1 


SHTJSYMTAB 

2 


SHT_STRTAB 

3 


Continued 
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Table 3.4 Continued 


Name 

Value 

SHT_RELA 

4 

SHT_HASH 

5 

SHT_DYNAMIC 

6 

SHTJMOTE 

7 

SHTJMOBITS 

8 

SHT_REL 

9 

SHT_SHLIB 

10 

SHT_DYNSYM 

11 

SHTJ.OPROC 

0x70000000 

SHT_HIPROC 

0x7FFFFFFF 

SHTJ.OUSER 

0x80000000 

SHTJHIUSER 

OxFFFFFFFF 


The SHT_NULL value indicates that the section header entry is inactive and does not 
have an associated section. The SHT_PROGBITS value indicates that the section holds data 
deYned by the program itself and that the format is only known to the program. The 
SHT_SYMTAB and SHT_DYNSYM sections deYne symbol tables. An application may 
have exactly one SHT_SYMTAB section (although likely it will have none). It typically 
contains a complete table of symbols, whereas the SHT_DYNSYM section deYnes a minimal 
set of symbols to be used for dynamic linking. Neither of these sections is gone into in 
detail since it is unlikely you will be reversing a Yle that actually has symbolsohowever, the 
interested reader is encouraged to read the ELF speciYcation. The SHT_STRTAB is a 
section that holds a string table; a Me can have more than one string table section and these 
sections are explained in more detail later on in this chapter. SHT_RELA sections contain 
relocation entries with explicit addends. SHT_HASH sections contain a symbol hash table 
and at the moment only one section of this type is allowed per object. This section is 
required by all objects participating in dynamic linking. The SHT_DYNAMIC type is used 
for dynamic linking. SHT_NOTE indicates that the section holds information that marks 
the Me in some way. This section is not described for brevityis sake. SHT_NOBITS indicates 
that the section occupies no space in the Me but otherwise resembles an SHT_PROGBITS 
section; the most well-known SHT_PROGBITS section is the .BSS. SHT_REL holds 
relocation entries without explicit addends, and the SHT_SHLIB section is reserved but has 
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unspeciYed semantics. SHT_LOPROC and SHT_HIPROC deYne a range of sections that 
are reserved for processor-speciYc semantics, whereas SHT_LOUSER and SHT_HIUSER 
are the same except they are reserved for the application. 

Following the shjtype Yeld we have the shjlags Yeld, which deYnes various 1-bit 
attributes for the section. The attributes are enumerated in Table 3.5. 


Table 3.5 Attributes 

Name 

Value 

SHF_WRITE 

0x1 

SHF_ALLOC 

0x2 

SHF_EXECINSTR 

0x4 

SHFJVIASKPROC 

OxFOOOOOOO 


The SHF_WRITE Dig indicates that the section will be writeable. SHF_ALLOC 
indicates that the section should actually reside in memory at run-time. SHF_EXECINSTR 
indicates that the section contains executable instructions. Finally SHF_MASPROC 
indicates that the section is reserved for processor-speciYc uses. 

Next we have the sh_addr Yeld; it provides the preferred VA at which the section should 
start. The sh_offset is similar, except that it provides a Yle offset from the beginning of the Yle 
to the beginning of the section. The sh_size Yeld is the size of the section, unless it is of type 
SHT_NOBITS, although a SHT_NOBITS section can have a nonzero sh_size ; it just does 
not occupy space in the physical Yle. The shjlink and sh_info members are related in that 
both values are subject to interpretation dependent on the section type. In the case of a 
SHT_DYNAMIC section, the shjlink member indicates the section header table index of 
the string table used by the section, and has a shjnfo Yeld of zero. For SHT_HASH the 
shjink member holds the section header table index of the symbol table that applies to the 
hash table, and again has an shjnfo value of zero. For a full list of values, please refer to the 
ELF speciYcation. Finally, the sh_entsize member applies to certain sections that have Yxed- 
size tables inside of them; for instance, given a section with a symbol table, this entry would 
indicate the length of the symbol table. 

Now that we have some comprehension of the various sections, Table 3.6 denotes sec¬ 
tions that are fairly common and standard, with their types, attributes and a brief description. 
It should be noted that sections whose name has a leading period (such as .bss) are reserved 
for use by the system. 
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Table 3.6 Common Sections 


Section Name 

Type 

Attributes 

Description 

.bss 

SHTJSIOBITS 

SHF ALLOC 
SHF_WRITE 

Holds uninitialized data, 
is initialized with zeros 
at load time; typically 
globally scoped 

.comment 

SHT_PROGBITS 

n/a 

Contains version 
control information 

.data 

SHT_PROGBITS 

SHF ALLOC 
SHF_WRITE 

Contains initialized 
data that contributes 
to applications memory 
image, typically 
globally scoped 

.datal 

SHT_PROGBITS 

SHF ALLOC, 
SHF_WRITE 

Contains initialized 
data that contributes 
to applications memory 
image, typically 
globally scoped 

.debug 

SHT_PROGBITS 

n/a 

Unspecified contents 
used for symbolic 
debugging 

.dynamic 

SHT_DYNAMIC 

SHF ALLOC 
SHF_WRITE 
(processor 
specific) 

Contains dynamic 
linking information, 
whether SHF_WRITE 
is specified or not is 
processor specific 

.dynstr 

SHT_STRTAB 

SHF_ALLOC 

Contains strings 
needed for dynamic 
linking 

.dynsym 

SHT_DYNSYM 

SHF_ALLOC 

Contains strings needed 
for dynamic linking 

.fini 

SHT_PROGBITS 

SHF ALLOC 
SHF_EXECINSTR 

Contains executable 
instructions used in 
application termina¬ 
tion, such as destructors 

■got 

SHT_PROGBITS 


Described in detail later 

.hash 

SHT_HASH 

SHF_ALLOC 

Contains a symbol 
hash table 

.init 

SHT_PROGBITS 

SHF ALLOC 
SHF_EXECINSTR 

The inverse of .fini, 
for ex., contains 
constructors 
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Table 3.6 Continued 


Section Name 

Type 

Attributes 

Description 

.interp 

SHT_PROGBITS 

SHF_ALLOC 

Holds the path name 
of a program 
interpreter; if the 
file contains a 
loadable segment 
then the SHF_ALLOC 
attribute will be set 

.line 

SHT_PROGBITS 

n/a 

Contains line 
information for 
debugging 

.note 

SHT_NOTE 

n/a 

Can be used by the 
implementation to 
allow stigmatic 
marking of an 
executable 

.pit 

SHT_PROGBITS 


Described in detail later 

,rel<name> 

SHT_REL 

SHF_ALLOC 

Contains relocation 
information; if there 
is a loadable 
segment then 
SHF_ALLOC will be 
set. Traditionally in 
the place of <name> 
is the name of the 
section that the 
relocations are for, 
such as .rel.text 

,rela<name> 

SHT_RELA 

SHF_ALLOC 

Contains relocation 
information; if there 
is a loadable 
segment then 
SHF_ALLOC will be 
set. Traditionally in 
the place of <name> 
is the name of the 
section that the 
relocations are for, 
such as .rela.text 


Continued 
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Table 3.6 Continued 


Section Name 

Type 

Attributes 

Description 

.rodata 

SHT_PROGBITS 

SHF_ALLOC 

Contains read-only 
data, such as 
constant strings 

.rodatal 

SHT_PROGBITS 

SHF_ALLOC 

Contains read-only 
data, such as 
constant strings 

.shstrtab 

SHT_STRTAB 

n/a 

Contains section 

names 

.strtab 

SHT_STRTAB 

SHF_ALLOC 

Contains strings, 
typically names 
associated with 
symbol table 
entries. If there is a 
loadable section 
then SHF_ALLOC will 
be specified. 

.symtab 

SHT_SYMTAB 

SHF_ALLOC 

Contains a symbol 
table (not described 
in this chapter). If 
there is a loadable 
section, then 

SHF_ALLOC will be 
specified 

.text 

SHT_PROGBITS 

SHF ALLOC 
SHF_EXECINSTR 

The executable 
instructions that 
make up the 
program 


The program header table is an array of program header structures; these structures deYne 
segments and generally deYne how to load the binary to the operating system (OS). The size 
of the table and the number of entries are speciYed in the ELF header. Some of the seg¬ 
ments are supplementary, whereas others contribute to the process image. Just like everything 
else in the ELF (sans the ELF header itself) Yle, there is no speciYc ordering of the segments 
nor speciYc offset to the program header table; this is deYned solely by the ELF header. In 
Figure 3.12 to the right you will Ynd a diagram detailing the ordering and members of an 
Elf32_Phdr structure. The p_type Yeld indicates what type of segment is being described and 
by implication tells the system how to interpret its contents. The deYned values are shown in 
Table 3.7. 
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Table 3.7 Defined Values 


Name 

Value 

PTJMULL 

0 

PT_LOAD 

1 

PT_DYNAMIC 

2 

PTJNTERP 

3 

PT_NOTE 

4 

PT_SHLIB 

5 

PT_PHDR 

6 

PT_LOPROC 

0x700000000 

PT_HIPROC 

0x7FFFFFFF 


Figure 3.12 Program Header Structure 


Offset 

Size 

Reid Name 

0 

4 

p type 

4 

4 

p offset 

8 

4 

p vaddr 

12 

4 

p_paddr 

16 

4 

p filesz 

20 

4 

pmemsz 

24 

4 

pflags 

32 

4 

palign 


Bold indicates discussed/referenced field 


Segments of type PT_NULL are unused; the values of its other members are undeYned 
(and thus should be ignored). The reasoning behind having another NULL type of segment 
is to allow segments to be deYned but ignored by the implementation. Segments of type 


www.syngress.com 



60 Chapter 3 • Portable Executable and Executable and Linking Formats 

PT_LOAD are actually loaded into memory preferably at the address p_vaddr. The Yrst 
p_filesz bytes at p_qffset from the Yleis memory-mapped base are loaded into memory 
If p_memsz is larger than p_filesz then these bytes are also mapped into the segment and 
zero Ylled; it is invalid for a p_fllesz member to be greater in value than p_memsz. The 
PT_INTERP segment, if present, must precede any PT_TOAD segments as it indicates the 
path name of the program interpreter. This segment can occur only once in the Me (if the 
Yle wishes to be valid). Segments of type PT_DYNAMIC are related to dynamic linking. 
PT_NOTE is relatively unimportant but allows interacting applications to check confor¬ 
mance (that is, GLIBC version). PT_SHLIB is deYned but reserved, and Yles containing a 
PT_SHLIB segment do not conform to the ABI. The PT_PHDR segment speciYes the size 
and location of the program header table. This speciYcation applies to both in the physical 
Yle and in the memory image. Like the PT_INTERP segment, it can only occur once and 
if present must occur before any PT_LOAD segments. Finally, PT_LOPROC and 
PT_HIPROC are reserved ranges for processor-speciYc functionality. 

As one might have guessed from the previous description, the p_offset member speciYes 
the offset of the segment from the beginning of the Yle. The p_vaddr member speciYes the 
preferred VA of the segment. The p_filesz and p_memsz elements specify the size of the 
segment in the physical Yle and in memory, respectively, and Ynally the p_ flags speciYes 
attributes of the segment. The three possible values are PF_R, PF_W and PF_X for read, 
write and execute, respectively. 

Now that we have some basic understanding of segments, ids possible to talk a bit about 
the differences between executable images, shared library images, or images that have address 
space layout randomization (ASLR) applied. Typically, in order to load an executable image, the 
address for each segment used when building the image must be included. This address is 
speciYed in the segments p_vaddr member. This is the result of the image having absolute 
references that would break if the addresses were changed. ASLR images and shared library 
images typically get around this restriction by using what is known as position independent code 
(PIC). The general idea behind PIC is that, instead of using absolute references to some 
piece of data, relative references are used. For instance, whereas in traditional code you may 
access a variable at address XYZ, in PIC you would reference that by another meansothat 
is, relative to your current position. In the case of an application using a shared library to 
access commonly used functions, such as is common with the standard C library, a series of 
intermediaries are usedoin the case of ELF, it is the global offset table (GOT or .got), the 
dynamic segment/section (_DYNAMIC or .dynamic) and the procedure linkage table (PLT or 
.pit). These three segments are integrally interrelated and make up one of the major reasons 
for adopting ELF over older standards such as a.outonamely, standards-supported dynamic 
linking. The .dynamic segment is present in every executable image that takes part in 
dynamic linking; this segment is referenced by the symbol _DYNAMIC, which is an array 
of structures as illustrated in Figure 3.13. 
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Figure 3.13 Dynamic Structure 


Offset 

Size 

Field Name 

0 

4 

d tag 

4 

4 

dval 

4 

4 

dptr 


Bold indicates discussed/referenced field 


The dynamic structure contains two values, a tag followed by a union. The tag 
determines how the union will be interpreted. The d_val member contains an integer with 
various interpretations that are described below, whereas the d_ptr member contains a VA. 

As you well know, the compile-time VA and the runtime VA might differ and the relocations 
section does not contain relocations for the _DYNAMIC array. In Table 3.8 you will Ynd 
several deYned d_tag types, and whether they are optional or mandatory. This is not the full 
list but only what is relevant to us, and the interested reader is again highly encouraged to 
refer back to the ETF speciYcation for a more detailed and complete explanation. 


Table 3.8 Defined djtag Types 


Name 

Value 

d_val or d_ptr? 

executable 

Shared object 

DT_NULL 

0 

ignored 

mandatory 

mandatory 

DT_PLTRELSZ 

2 

d_val 

optional 

optional 

DT_PLTGOT 

3 

d_ptr 

optional 

optional 

DT_FINI 

13 

d_ptr 

optional 

optional 

DT_PLTREL 

20 

d_val 

optional 

optional 

DTJMPREL 

23 

d_ptr 

optional 

optional 


The DT_NULL element marks the end of the _DYNAMIC array, and thus it is a nec¬ 
essary element. Aside from this element, there is no inherent ordering within the array. The 
DT_PTTRETSZ element holds the total size of relocation entries associated with the PET. 

If a DT_JMPREE element is present, then a corresponding DT_PTTREESZ entry must also 
be present. DT_PLTGOT entries hold an address associated with either the GOT or the 
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PLT, both of which are described in further detail below. DT_FINI elements hold the 
address of a termination function, or a destructor, which potentially would be useful to 
hackers and therefore is of interest to us and is covered brieEy later on. Finally, the 
DT_JMPREL entry contains a pointer to relocation entries associated only with the PLT. 
Separating these relocations allows the linker to ignore them during image initialization and 
use a form of linking known as lazy binding. Quite simply, lazy binding defers the relocation 
until the actual use of that symbol. For instance, consider the following C program: 

#include <stdio.h> 

#include <stdlib.h> 
int 

main(void) 


unsigned int cnt; 
for (cnt = 0; cnt <2; cnt++) 
printf("cnt: %u\n", cnt); 
exit(EXIT SUCCESS); 


In this application, we have two (visible) standard library functions called printf() and 
exit(). In a lazy binding scenario, neither symbol is resolved until the last possible minute. 

The Yrst time printf() is called the symbol is resolved, incurring the overhead of relocation 
then, instead of at initialization. However, the second time printf() is called, the symbol has 
already been resolved and the overhead is not incurred to resolve printf() again. However, that 
overhead is again incurred to resolve exit(). The advantage of this, of course, is the increase in 
speed and efYciency; after all, not every symbol is going to be resolved. Furthermore, it 
makes dynamic module loading easier to cope with. However, the downside of this comes in 
the form of attack surfaceoit makes a buggy program more easily exploited. In recent years, 
a Ehg passed to hardened GCC tool-chains called relro does all relocations at program initial¬ 
ization and then disallows writing to the segments at runtime. This means that a potential 
attacker cannot take advantage of some dubious pointer arithmetic or a buffer overHbw and 
write into these sections and then return execution Ebw back into these sections. Typical 
sections to be marked this way are .init/.ctors, .Yni/.dtors, the PLT, GOT and .dynamic, 
although speciYcs depends upon architecture. This methodology is becoming more and more 
common, and as a reverse engineer it is likely that as time progresses the chances of your 
running across lazy binding decreases. 

The global offset table, which on IA32 platforms is accessible under the symbol 
_GLOBAL_OFFSET_TABLE, is an array of addresses. These addresses are absolute references 
and allow the position-independent code to have relative references. Thus, the PIC code will 
obtain the address of the GOT and extract absolute references from its relative ones. The 
symbol _GLOBAL_OFFSET_TABLE need not refer to the beginning of the .got segment, 
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and thus negative and positive indexes are potentially valid. When the image is loaded, the 
dynamic linker walks through the relocations and looks for entries of a speciYc type and 
replaces their entries in the GOT with their absolute addresses, effectively getting around the 
limitations of the static linker. The Yrst element of the GOT is a special entry that contains 
the address of the _DYNAMIC structure; this allows the dynamic linker to process the GOT 
by Ynding itself in the _DYNAMIC structure without having to depend on any relocations. 
Furthermore, on IA32 the second and third entries are also reserved to have special values. 
The GOT redirects position-independent addresses to absolute locations, whereas the PLT 
does the same but for functions. The PLT determines the functionis absolute address and 
updates the GOT as necessary. The exact implementation of the PLT varies, depending on 
whether it was compiled PIC or not. A non-PIC entry looks something like the following: 

PLT 
. PLTO 

push address_of_GOT+0x04 

jmp [address_of_GOT+0x08] 

nop 

nop 

nop 

nop 

.PLT1 : 

jmp [namel_in_GOT] 

push offset 

jmp [,PLT0+$] 

.PLT2 : 

jmp [name2_in_G0T] 

push offset 

jmp [,PLT0+$] 

whereas a PIC PLT might look something like this: 

PLT 

.PLTO 

push [ebx+0x04] 

jmp [ebx+0x08] 

nop 

nop 

nop 

nop 
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. PLTl 


jmp [ebx+namel] 

push offset 

jmp [.PLT0+$] 

. PLT2 

jmp [ebx+name2] 

push offset 

jmp [.PLT0+$] 


With all this taken into context, in order to resolve dynamic references, the linker and 
the application work in tandem according to the following steps: 

0. Upon creation of the image, the second and third entries in the GOT are set to 
their special values as deYned below. 

1. If the PLT is PIC, then the address of the GOT must reside in the ebx register. 

The calling function is responsible for placing the address into this register. 

2. Assume that the application is trying to call name I which can be found in the label 
.PLTl. 

3. The Yrst instruction under that label is a jump into the GOT, which initially 
contains the address of the push and jmp instructions following the jump instruction 
into the GOT. 

4. The application then pushes the address of the relocation entry, represented in this 
case by the variable named offset. This offset will specify GOT entry used in the 
prior jump along with a symbol table index, namei in this instance. 

5. The application then jumps to .PLTO and pushes the address of the second element 
of the GOT onto the stack, giving the dynamic linker a word to reference for 
identification purposes, and then transfers control to the third GOT entry, which 
hands control to the dynamic linker. 

6. The dynamic linker unwinds the stack and retrieves the identifying information, 
Ynds the absolute address for the symbol and stores it in the related GOT entry, 
and then hands control to the requested function. 

7. Further calls to this function will skip the push of offset and will jump to .PLTO as a 
result of having the GOT entry modiYed. 

So, as you can see, dynamic linking is accomplished by indirection and abstraction. The 
application doesnit know beforehand exactly what address itis calling, and in turn calls into 
the PLT, which in turn jumps to the GOT; if the address has not already been resolved, 
control is handed back into the PLT, which pushes the relocation entry and jumps to the 
Yrst entry in the PLT, which then hands control to the dynamic linker. 
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Note 

As previously mentioned, several advances have been made in the not-so- 
distant past that require that lazy binding be turned off so that relocations 
can occur at initialization instead of at runtime. If you think hard enough, 
now that you know how the GOT/PLT works, you might realize why. If as an 
attacker I can overwrite a GOT entry, then it's really only a matter of the 
application calling that function again before my shellcode obtains control. 
This technique has been documented in several places; one of the white 
papers can be found at the following URL: www.milwOrm.com/papers/3. 

Similarly, a given image destructor can be attacked by overwriting data in 
.dtors, which typically contains the addresses of functions in .fini. By overwrit¬ 
ing an address there, it becomes potentially possible for the application to 
have a rogue function called upon program execution. This technique was 
documented by Juan M. Bello Rivas and his white paper can be found at the 
following URL: http://synnergy.net/downloads/papers/dtors.txt 
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Summary 

In conclusion, weive taken you on a brief tour of the PE and ELF Me formats, giving you a 
basic understanding of these formats and hopefully giving you the knowledge needed to 
perform limited manual analysis of either one. You should be able to determine the imports 
and exports of a PE (and consequently have a basic understanding for rebuilding the imports 
section in a packed PE), and have a decent understanding of how dynamic linking occurs in 
ELF YLes. In both formats, you should be fairly comfortable with their members and struc¬ 
ture and generally have some understanding of how a linker and loader operate on either of 
the formats. As suggested throughout the chapter, the reader is encouraged to read the origi¬ 
nal speciYcations themselves, as this chapter is far from complete and tries to emphasize only 
elements that the author thought would be most important. Some concepts that you might 
Ynd particularly interesting or useful may not be covered. The ELF speciYcation can be 
found at the following URL: www.muppetlabs.com/~breadbox/software/ELF.txt or via your 
favorite search engine by searching for iELF speciYcation! The PE speciYcation can be 
obtained from Microsoftis website at www.microsoft.com/whdc/system/platform/Yrmware/ 
PECOFF.mspx. 
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Introduction 

In this chapter we will step away from the basics of IDA and dive straight into applying our 
knowledge. This is a good starting point for the average computer or security professional 
with a general knowledge of security, assembly basics and programming. We will begin by 
figuring out exactly what our first example binary does, and then move to applying this 
knowledge in common practices within the security industry. Specifically, we’ll see if we can 
find the password it’s asking for when it first starts and then leverage this knowledge in order 
to find vulnerability within the binary. Applying these two approaches, we’ll finally be able 
to understand the steps needed to actually exploit the application. 

The example code and binaries we will be using for this chapter are available for 
download from the Syngress website. The download file is called StaticPasswordOverflow.zip. 

Following Execution Flow 

The first step in reversing any binary on the planet is determining exactly what it is doing 
and how it is doing it. Tet’s jump into the immediate task of following the instructions of 
our application step by step, and take notes on the general operations within the binary. To 
begin, let’s go straight ahead into the first useful chunk of code. Although it’s personal pref¬ 
erence, I prefer using a notepad or notebook of some sort so I can keep my thoughts as I 
move along, write down addresses, and generally keep track of everything. You never know 
when you might need a note from the beginning of your reversing, and typing numbers has, 
for me, always been much slower than writing them down. Plus, it’s much easier to draw 
pictures on paper! 

.text:00401270 ; int _cdecl main(int argc,const char **argv,const char *envp) 


text:00401270 

Dst 

= byte ] 

ptr -80h 

text:00401270 

argc 

= dword 

ptr 8 

text:00401270 

argv 

= dword 

ptr OCh 

text:00401270 

envp 

= dword 

ptr lOh 

text:00401270 


push 

ebp 

text:00401271 


mov 

ebp, esp 

text:00401273 


sub 

esp, 80h 

text:00401279 


push 

offset aReverseEnginee 

text:0040127E 


call 

sub_401554 

text:004 012 83 


add 

esp, 4 

text:00401286 


push 

offset aPleaseProvideT 

text:004 012 8B 


call 

sub_401554 

text:00401290 


add 

esp, 4 

text:004 012 93 


push 

80h ; Size 

text:00401298 


push 

0 ; Val 
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text:004 012 9A 

lea 

eax, [ebp+Dst] 

text:0040129D 

push 

eax ; Dst 

text:004 012 9E 

call 

memset 

text:004012A3 

add 

esp, OCh 

text:004012A6 

lea 

ecx, [ebp+Dst] 

text:004012A9 

push 

ecx 

text:004012AA 

push 

offset al27s ; "%127 

text:004012AF 

call 

scant 

text:004012B4 

add 

esp, 8 

text:004012B7 

lea 

edx, [ebp+Dst] 

text:004012BA 

push 

edx ; Str2 

text:004012BB 

call 

sub_4011C0 

text:004 012C0 

add 

esp, 4 

text:004 012C3 

movsx 

eax, al 

text:004 012C6 

test 

eax, eax 

text:004 012C8 

jge 

short loc 4012D9 

text:004012CA 

push 

offset aYouFailed 

text:004012CF 

call 

sub_401554 

text:004012D4 

add 

esp, 4 

text:004012D7 

jmp 

short loc 4012E6 

text:004012D9 

loc 4012D9: 

; CODE XREF: main+58 

text:004012D9 

push 

offset aYouWon Goodbye 

text:004012DE 

call 

sub_401554 

text:004012E3 

add 

esp, 4 

.text:004012E6 

loc 4012E6: 

; CODE XREF: main+67 

text:004012E6 

mov 

eax, 1 

text:004012EB 

mov 

esp, ebp 

text:004012ED 

pop 

ebp 

text:004012EE 

retn 


text:004012EE 

main endp 



At a glance, we can see that the main function doesn’t really do much of anything. It has 
a few calls and a few conditional statements. Also, just from the strings within some of these 
statements, it looks like we can assume there is a success/failure statement within this code; 
thus, the strings containing “YouFailed” and “YouWon”. We could switch over to the 
graph view right away to determine how these conditionals work, but first we will get an 
understanding for how this entire function works, so we have no surprises later on. 


text:004 0127 9 

push 

offset aReverseEnginee 

text:0040127E 

call 

sub_401554 

text:00401283 

add 

esp, 4 
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.text:00401286 push offset aPleaseProvideT 

.text:0040128B call sub_401554 

Here we can see that, after setting up the stack, it’s pushing some static strings into the 
buffer and calling a function. By looking at the strings, it’s safe to assume this is some sort of 
startup header printing. However, it looks like IDA cannot determine exactly what function 
this binary is calling. Let’s go ahead and follow the call and see where it’s going; select the 
call instruction and press Enter to jump to that location. 

. text: 00401554 ; int printf (const char *,...) 

.text:00401554_printf proc near ; CODE XREF: sub_401000+65 

.text:00401554; sub_401000+C0 

Whoops! It looks like IDA didn’t want to identify what exactly this function was; it’s just 
a statically compiled printf. It’s fairly safe to assume this function isn’t doing anything odd or 
funky right now, so we’ll chalk that one up as a simple print function and get on with it. 
Let’s press the Backspace key in order to get back to our entry point and continue. 


Note 

Some functions may not always be what they appear to be; always give such 
obviously named static functions a good look before assuming the name is 
real. You never know what the bad guys might be trying to hide with clever 
names. 


Since we know what that routine is, we should quickly rename it within IDA so we 
don’t have to worry about it confusing us later. The easiest way to accomplish this is to 
simply click the name of the instruction, and press the N key, which will pop up the rename 
dialog box. For the sake of ease, we’ll rename this function printf since that is what it really 
is. Now that we have that out of the way and have confirmed that it is just printing strings 
as a sort of startup process, we’ll continue down the code. 


text:00401290 

add 

esp. 

4 

text:00401293 

push 

80h 

; Size 

text:00401298 

push 

0 

; Val 

text:004 012 9A 

lea 

eax, 

[ebp+Dst] 

text:0040129D 

push 

eax 

; Dst 

text:004 012 9E 

call 

memset 

text:004012A3 

add 

esp. 

OCh 

text:004012A6 

lea 

ecx, 

[ebp+Dst] 
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text:004012A9 

push 

ecx 


text:004012AA 

push 

offset al27s ; 

■ " % 12 7 s 

text:004012AF 

call 

scant 



Stepping through this set of instructions, it seems obvious that it is calling memset() to fill 
a buffer, and then using scanf() in order to read into a buffer. Specifically, we can see that the 
memset() call is filling the first 0X80, or 128, bytes of the Dst stack buffer with 0x00, or 
NULL. This can be deduced by seeing the values being pushed prior to the call, where we 
see the following four instructions: 


text:00401293 

push 

80h 

; Size 

text:00401298 

push 

0 

; Val 

text:004 012 9A 

lea 

eax, 

[ebp+Dst] 

text:0040129D 

push 

eax 

; Dst 


We can see here that the binary is pushing a size of 0x80, a value of 0 and finally push¬ 
ing a pointer to the address of [ebp+Dst], our stack variable. Finally, we can also see that 
these same operations are being used for the scanf() call. Specifically, the instruction to load a 
pointer to the Dst buffer (lea ecx, [ebp+Dst]) is performed again, and the result of scanf() is 
saved within this buffer. We can also see that the scanf() call is correctly filling the buffer, 
with the %ol21s definition for its format string: thus only saving a maximum of 127 bytes 
to the buffer. 


Note 

If you don't feel comfortable enough with 16-base hex numbers yet, you can 
always right-click a numerical value within IDA and view or select a different 
base type for the numeral. Although IDA's default is hex values, you can click 
the value and press the H key in order to switch it to standard 10-base 
numbers. 


Momentarily going back to the stack initialization portion of this function, we can 
check to make sure these sizes correspond with the actual size of the operations being per¬ 
formed on this variable. We can see IDA has already determined that this function did have a 
variable, and its size was 0x80 bytes long. Additionally, we can see the stack initialization calls 
performing this, thus confirming that this is the hard set size of this stack variable. 

.text:00401270 Dst = byte ptr -80h 


.text:00401273 sub esp, 80h 
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Now we have the uninteresting portions of the code out of the way and we understand 
what it all does. We can finally move forward to the interesting conditionals we saw within 
the code, which seem to be where all the magic must be happening. If you switch over to 
the graphing view now (press the Spacebar) you can see the conditional jumps that occur 
right after the sprintf() call, as shown in Figure 4.1. Additionally, you can see the mini-graph 
window, which becomes extremely useful with larger functions with many conditional 
jumps. 


Figure 4.1 Graphing View 
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lea 
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push 

call 
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lea 

push 

call 

add 

nousx 

test 

jge 


eax 

nemset 

esp, OCh 

ecx, [ebp+Dst] 

ecx 

offset a127s 
seanf 

esp, 8 

edx, [ebp*Dst] 
edx 

subJiOIICO 
esp, *i 
eax, al 
eax, eax 

short loc 'I012D9 


; Dst 


••t127s** 


BN i.4 


push 

offset avoufailed uooa 

”Vou failed, uooooye.\n” 


call 

print! 



add 

esp, 4 



jnp 

short loc_UU 12 L 6 




H N id 
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Reversing What the Binary Does 

Moving past what seems to be the setup portion of this function, we can see that the 
function is calling a few subroutines prior to the conditional jump that has become our goal. 
Specifically, we can see that, prior to the conditional, the subroutine is calling another 
routine that is statically within the binary, and then immediately shifting the stack and 
performing the conditional. 
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Binary Subroutines in IDA Pro 

Be careful when analyzing subroutines within binaries that don't appear to be readily 
identifiable. Although it is commonly safe to assume that these are part of the actual execut¬ 
able itself, IDA Pro will identify many statically compiled libraries in this manner as well. 

Always be sure to import debug symbols when possible and to label all functions 
once you have a good understanding of their purpose. ELF linux binaries are notori¬ 
ous for this, and many countless hours can be lost tracing statically compiled base 
libraries or GOT tables. 


text:004012A9 

push 

text:004012AA 

push 

text:004012AF 

call 

text:004012B4 

add 

text:004012B7 

lea 

text:004012BA 

push 

text:004012BB 

call 

text:004 012C0 

add 

text:004 012C3 

movsx 

text:004 012C6 

test 

text:004 012C8 

jge 


ecx 

offset al27s ; "%127s 

_scanf 

esp, 8 

edx, [ebp+Dst] 

edx ; Str2 

sub_4011C0 

esp, 4 

eax, al 

eax, eax 

short loc 4012D9 


We can move through the code immediately after the scanf() function call, which seems to 
handle the return value of scanf() and then prepare the stack to call this mysterious subroutine. 
Assuming you already have a good understanding of the stack structure and function call 
methods, let’s move through this again as review. Two values are pushed just prior to the sccmf() 
call, one of which is a static string within our binary. Using documentation available from 
many sources, we can easily deduce not only that these are the variables for scanf() being 
pushed, but also what they mean. This is extremely useful when reversing binaries that use 
less-common library functions. It will always be a generally good idea to look up library func¬ 
tions prior to attempting to reverse them; if they aren’t critical to our goal, it can save lots of 
time to assume they perform as advertised. Below, you can see that scanf() is structured as such. 

int scanf (const char * format, ...) ; 


Noting this, we can now see that what is actually being loaded and passed is first the 
pointer to a buffer, and then the format string (always remember, variables are pushed into the 
stack “backwards”). Next, the stack is shifted 8 bytes, and then the pointer to our buffer 
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[ebp+Dst] is loaded into edx and then pushed onto the stack. After this, our magical subroutine 
is simply called. It will be safe to assume now that this function is performing some sort of 
mystical processing on the buffer being filled by scanf( ), and returning an integer value which is 
then compared. This can be deduced by the instructions immediately after the call: movsx eax, 
al and test eax, eax. These instructions tell us that this portion of code is taking the return value 
of the called subroutine, call sub_4011C0, and conditionally jumping if it is greater than or 
equal to (the return value of calls are generally provided within the eax register). 

So, as review, we now know that this mysterious subroutine is performing some sort of 
operation on our input value and providing the value that controls our conditional jump. We 
are getting closer to our goal! Now if we can discover what sort of operation this routine is 
performing and how to provide it with the correct value, we can control the conditional 
jump operation. For reference, let’s label the call statement input_process and then switch our 
IDA Pro view to that function by double-clicking its name. 

The Processing Subroutine 

Now that we know this function is going to be what inevitably controls our success or 
failure within the application, let’s step through the code in detail in order to understand 
what may be going on here. Additionally, looking at the code below, we will need to jump 
around a bit in order to truly understand what is happening. Not all reverse engineering can 
be performed by analyzing the executable from start to finish; analyzing end processing can 
sometimes yield a faster understanding of how everything got there. 

.text:004011C0 ; int _cdecl input_process(char *Str2) 

.text:004011C0 input_processproc near ; CODE XREF: _main+4B 

.text:004011C0 Dst = byte ptr -80h 

.text:004011C0 var_7F= byte ptr -7Fh 

.text:004011C0 var_7E= byte ptr -7Eh 

.text:004011C0 var_7D= byte ptr -7Dh 

.text:004011C0 var_7C= byte ptr -7Ch 

.text:004011C0 var_7B= byte ptr -7Bh 

.text:004011C0 var_7A= byte ptr -7Ah 

. text:004011C0 var_79= byte ptr -79h 

.text:004011C0 var_78= byte ptr -78h 

.text:004011C0 var_77= byte ptr -77h 

.text:004011C0 var_76= byte ptr -76h 

.text:004011C0 var_75= byte ptr -75h 

.text:004011C0 var_74= byte ptr -74h 

.text:004011C0 var_73= byte ptr -73h 

. text:004011C0 var_72= byte ptr -72h 

.text:004011C0 var_71= byte ptr -71h 

.text:004011C0 var_70= byte ptr -70h 
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. text:004011C0 Str2 = dword ptr 8 


text:004011C0 

push 

ebp 



text:004011C1 

mov 

ebp, esp 



text:004011C3 

sub 

esp, 80h 



text:004011C9 

push 

8 Oh 

Size 


text:004 01ICE 

push 

0 

Val 


text:004011D0 

lea 

eax, [ebp+Dst] 


text:004011D3 

push 

eax ; 

Dst 


text:004011D4 

call 

memset 



text:004011D9 

add 

esp, OCh 



text:004011DC 

mov 

[ebp+var 

70] , 

0 

text:004011E0 

mov 

[ebp+var 

75] , 

73h 

text:004011E4 

mov 

[ebp+Dst] 

, 7 4h 

text:004011E8 

mov 

[ebp+var 

76] , 

73h 

text:004011EC 

mov 

[ebp+var 

7F] , 

68h 

text:004011F0 

mov 

[ebp+var 

7A] , 

6Dh 

text:004011F4 

mov 

[ebp+var 

7C] , 

69h 

text:004011F8 

mov 

[ebp+var 

7B] , 

73h 

text:004011FC 

mov 

[ebp+var 

7 1 ] , 

64h 

text:00401200 

mov 

[ebp+var 

74] , 

77h 

text:00401204 

mov 

[ebp+var 

7E] , 

69h 

text:00401208 

mov 

[ebp+var 

7D] , 

73h 

text:0040120C 

mov 

[ebp+var 

78] , 

70h 

text:00401210 

mov 

[ebp+var 

73] , 

6Fh 

text:00401214 

mov 

[ebp+var 

72] , 

72h 

text:00401218 

mov 

[ebp+var 

79] , 

7 9h 

text:0040121C 

mov 

[ebp+var 

77] , 

61h 

text:00401220 

mov 

ecx, [ebp+Str2 

] 

text:00401223 

push 

ecx ; 

Str2 


text : 00401224 

lea 

edx, [ebp+Dst] 


text:00401227 

push 

edx ; 

Str 1 


text:00401228 

call 

strcmp 



text:0040122D 

add 

esp, 8 



text:00401230 

test 

eax, eax 



text:004 01232 

jz 

short loc 401247 

text:004 01234 

push 

offset alnvalidPas; 

\n******* INVALID 

PASSWORD 

Vc Vc ^ ^ 


text:00401239 

call 

printf 



text:0040123E 

add 

esp, 4 



text:004 01241 

or 

al, OFFh 



text:00401243 

jmp 

short loc 40125D 
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.text:00401245 jmp short loc_40125D 

.text:00401247 loc_401247: ; CODE XREF: input_process+72 

.text:00401247 mov eax, [ebp+Str2] 

. text:0040124A push eax 

.text:0040124B push offset aSIsCorrect_ ; "%s is correct.\n\n" 

.text:00401250 call printf 

. text:00401255 add esp, 8 

.text:00401258 call sub_401000 

.text:0040125D loc_40125D: ; CODE XREF: input_processt83 

.text:0040125D; input_processt85 

. text:0040125D mov esp, ebp 

. text:0040125F pop ebp 

.text:00401260 retn 

.text:00401260 input_process endp 

We can see here that this function is a bit larger than the entrypoint, but it appears that 
IDA Pro has helped that to an extent. Briefly looking at the code, something immediately 
jumps out as an important portion of the function and as something we should begin to 
look at. At the address 00401228, there is a strcmp() call. 


text:00401220 

mov 

ecx. 

[ebp+Str2 

text:00401223 

push 

ecx 

; Str2 

text:00401224 

lea 

edx. 

[ebp+Dst] 

text:00401227 

push 

edx 

; Str 1 

text:00401228 

call 

strcmp 


Not only does it appear to be a strcmp() call to compare strings, but it is using the buffer 
that stores the input from scanf(), ebp+Dst. This looks promising for our goal: a string com¬ 
parison against our input value, followed by a conditional statement that determines success 
or failure. But what can it be comparing against? Let’s step back a bit in the code and take a 
look. In this string comparison, we can see the function is comparing the pointers in ecx and 
edx, which are respectively ebp+Str2 and ebp+Dst. Before actually determining what Str2 is, 
let’s rename it for reference to StrPassword. 

Just prior to the comparison, there is a major chunk of mov instructions being performed 
on what appears to be a sequential segment of memory, which is actually within our 
preallocated buffers. A 128-byte buffer, or 0x80 bytes, is being allocated on the stack and 
being NULLed out by the memset() function. 


text:004011C0 

push 

ebp 



text:004011C1 

mov 

ebp. 

esp 


text:004011C3 

sub 

esp. 

80h 


text:004011C9 

push 

80h 

; 

Size 

text:004 01ICE 

push 

0 

; 

Val 
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text:004 01IDO 

lea 

eax, [ebp+Dst] 

text:004011D3 

push 

eax ; Dst 

text:004011D4 

call 

memset 


After this buffer is prepared, it is sequentially filled with these static variables. Upon 
closer inspection, however, it seems all these values are relatively close to each other numeri¬ 
cally as well. Not only that, but they actually appear to be within the ASCII range of charac¬ 
ters, which would make sense since they are being used in a string comparison. Now, let’s 
figure out what this value actually is! Sadly, when statically analyzing a binary we do not 
have the option of popping it open in a debugger and waiting for the value to be filled, 
so it will be best to just break apart the values by hand and determine what they are. 


Most reverse engineering experts will frequently use pencil and paper when 
actually diving into a binary. Taking notes, quickly mapping paths or values, 
and many other things are always going to be faster on paper as long as we 
still have keyboards on computers. Try to always keep a pencil and paper 
next to you so you can quickly note addresses, values and other random 
items; our memories are never going to be perfect. 


However, before we get further into it, here is a small anomaly within IDA’s interpreta¬ 
tion of this chunk of code that is interesting. In the middle of all our mov instructions, there 
is one mov that IDA has determined is writing to our input buffer. 

.text:004011E4mov [ebp+Dst], 74h 

How could this be? Peeking back at the variables IDA has determined this function has, 
we can see it has attempted to automatically assign a different variable reference for every sin¬ 
gle mov instruction except for this anomaly. At a glance, this really does make the binary seem 
more complex and confusing. Because of the way IDA Pro disassembles functions, it sometimes 
has difficulty internally determining the actual structure of memory. Furthermore, it is a gener¬ 
ally good rule of thumb not to totally rely on IDA Pro’s dissemination of variables and argu¬ 
ments; instead, use them as references at a glance when required. This anomaly is due to these 
types of issues that IDA has; it hasn’t appropriately taken into account stack alignment adjust¬ 
ments in its dissemination of the variables. So, ebp+dst, as IDA has determined it to be, would 
also be var_80 if IDA named it as such. We know a pointer was pushed into the stack prior to 
the function being called, and because of this IDA has not appropriately accounted for the 
stack shift inherent in this operation, thus rendering this misleading piece of code. Knowing 
how the stack is structured, we can determine how it looks as follows: 

[Buffer Created] 

[Pointer to Our Input String] 
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[Stack Pointer Save and Return Address] 

[Real Input String] 

Therefore, this instruction, which IDA says is overwriting our pointer, is actually writing the 
first byte of our character array, and thus is the first character of our password string. Now that 
we have taken note of that small problem and can appropriately account for it, we can sit down 
and figure out what this character array actually contains and what the password is. See Table 4.1 


Note 

When performing static analysis on binaries, it never hurts to map out the stack 
in a table on paper as you work through a function. Especially in more complex 
functions, we can't completely rely on IDA to get it right every time. This is why 
understanding actual execution and stack structure is even more important in 
static analysis. We have to infer how the executable is going to behave every 
step of the way without having the ability to test and verify our assumptions. 


Table 4.1 Password Dissection 


Position 

Value 

Character 

70 

0 

(NULL) 

71 

64 

d 

72 

72 

r 

73 

6F 

o 

74 

77 

w 

75 

73 

s 

76 

73 

s 

77 

61 

a 

78 

70 

P 

79 

79 

y 

7A 

6D 

m 

7B 

73 

s 

7C 

69 

i 

7D 

73 

s 

7E 

69 

i 

7F 

68 

h 

80 

74 

t 
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We can see now that this is in fact a valid ASCII string that is being compared against 
our input buffer. Since this is a generic x86 binary, of course it is being assigned backwards 
in code. Therefore, our password is “thisismypassword”. Going back and looking at the code, 
we can see that the conditional within this function is using the result of strcmpQ in order to 
determine whether we provided the correct password or not. We have now passed the first 
hurdle in determining what this application does and how it is protecting itself. 

Moving forward in the code, we can see that there is actually a check for the correct 
password using a conditional jump, followed by a final call to an external function prior 
to returning. Even more interesting, it seems that the return value of the called function, 
sub_40100, is passed through as the return value for this function as well. 

.text:00401255 add esp, 8 

.text:00401258 call sub_401000 

.text:0040125Dloc_40125D: ; CODE XREF: input_process+83 
.text:0040125D; input_process+85 Oj 
.text:0040125D mov esp, ebp 

.text:0040125F pop ebp 

.text:00401260 retn 

How is this determined? Just prior to the call to the function sub_401000, the stack is 
actually shifted by 8 bytes. More specifically, ESP is incremented. This moves the stack 
appropriately so that the actual stack location of both functions’ return values will be the 
same, thus passing the value through. 

Stepping back for a moment, we now know a few new bits of information about the 
binary. First, we now know we need a password in order to progress anywhere. Secondly, if 
we do enter the correct password, it will progress to a second function, which is the final 
return value given to the entrypoint main function. In order to get a successful response, 
then, we will be required to meet whatever conditions exist within the third function as 
well. On that note, let’s rename the call to SecondCheck and double-click it to view it. 

.text:00401000 SecondCheck proc near ; CODE XREF: input_process+98 
.text:00401000 Dst = byte ptr -4D0h 
.text:00401000 var_450 = byte ptr -450h 

.text:00401000 Dest = byte ptr -400h 
.text:00401000 push ebp 

.text:00401001 mov ebp, esp 

.text:00401003 sub esp, 4D0h 

.text:00401009 push esi 

.text:0040100A push edi 

.text:0040100B mov ecx, 13h 

.text:00401010 mov esi, offset aPleaseSelectAn ; "Please select an option 

from the follow"... 

.text:00401015 lea edi, [ebp+var_450] 
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text:0040101B 

rep movsd 


text:0040101D 

movsw 



text:0040101F 

movsb 



text:00401020 

push 

80h 

; Size 

text:00401025 

push 

0 

; Val 

text:00401027 

lea 

eax. 

[ebp+Dst] 

text:0040102D 

push 

eax 

; Dst 

text:0040102E 

call 

memset 

text:00401033 

add 

esp. 

OCh 

text:00401036 

push 

80h 

; Size 

text:0040103B 

push 

0 

; Val 

text:0040103D 

lea 

ecx, 

[ebp+Dest] 

text:00401043 

push 

ecx 

; Dst 

text:00401044 

call 

memset 

text:004 0104 9 

add 

esp. 

OCh 

text:0040104C 

loc 40104C: 

; CODE XREF: SecondCheck+lAB 

text:0040104C 

mov 

edx. 

1 

text:00401051 

test 

edx. 

edx 

text:00401053 

jz 

loc 

4011B0 

text:00401059 

lea 

eax. 

[ebp+var 450] 

text:0040105F 

push 

eax 


text:00401060 

push 

offset aS ; "%s" 

text:00401065 

call 

printf 

text:0040106A 

add 

esp. 

8 

text:0040106D 

lea 

ecx. 

[ebp+Dst] 

text:00401073 

push 

ecx 


text:00401074 

push 

offset al27s 0 ; "%127s" 

text:004 0107 9 

call 

scant 

text:0040107E 

add 

esp. 

8 

text:00401081 

push 

80h 

; Count 

text:00401086 

lea 

edx. 

[ebp+Dst] 

text:0040108C 

push 

edx 

; Source 

text:0040108D 

lea 

eax. 

[ebp+Dest] 

text:00401093 

push 

eax 

; Dest 

text:00401094 

call 

strncat 

text:00401099 

add 

esp. 

OCh 

text:0040109C 

push 

offset Str2 ; "Exit" 

text:004010A1 

lea 

ecx. 

[ebp+Dst] 

text:004010A7 

push 

ecx 

; Str 1 

text:004010A8 

call 

strcmp 
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. text:004010AD add esp, 8 

. text:004010B0 test eax, eax 

. text:004010B2 jnz short loc_4010CD 

. text:004010B4 lea edx, [ebp+Dst] 

. text:004010BA push edx 

. text:004010BB push offset aOperationSComp ; "Operation: %s: Completed\n" 

.text:004010C0 call printf 

. text:004010C5 add esp, 8 

. text:004010C8 jmp loc_401195 

. text:004010CD loc_4010CD: ; CODE XREF: SecondCheck+B2 

.text:004010CD push offset aSelect ; "Select" 

. text:004010D2 lea eax, [ebp+Dst] 

. text:004010D8 push eax ; Strl 

. text:004010D9 call _strcmp 

. text:004010DE add esp, 8 

. text:004010E1 test eax, eax 

. text:004010E3 jnz short loc_4010FE 

. text:004010E5 lea ecx, [ebp+Dst] 

. text:004010EB push ecx 

. text:004010EC push offset aOperationSCo_0 ; "Operation: %s: Completed\n" 

. text:004010F1 call printf 

. text:004010F6 add esp, 8 

. text:004010F9 jmp loc_401195 

. text:004010FE loc_4010FE: ; CODE XREF: SecondCheck+E3 
. text:004010FE push offset aDrop ; "Drop" 

. text:00401103 lea edx, [ebp+Dst] 

. text:00401109 push edx ; Strl 

. text:0040110A call _strcmp 

. text:0040110F add esp, 8 

. text:00401112 test eax, eax 

. text:00401114 jnz short loc_40112C 

.text:00401116 lea eax, [ebp+Dst] 

. text:004 0111C push eax 

. text:0040111D push offset aOperationSCo_l ; "Operation: %s: Completed\n" 

. text:0040112 2 call printf 

. text:0040112 7 add esp, 8 

. text:0040112A jmp short loc_401195 

. text:004 0112C loc_40112C: ; CODE XREF: SecondCheck+114 

. text:0040112C push offset aCreate ; "Create" 

. text:00401131 lea ecx, [ebp+Dst] 
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.text = 004 01137 


push 

ecx ; Strl 

.text:004 0113 8 


call 

strcmp 

.text:004 0113D 


add 

esp, 8 

.text:004 0114 0 


test 

eax, eax 

.text = 004 0114 2 


j nz 

short loc 40115A 

.text:004 0114 4 


lea 

edx, [ebp+Dst] 

.text:004 0114A 


push 

edx 

.text:004 0114B 


push 

offset aOperationSCo 2 ; "Operation: %s: CompletedV 

.text = 004 0115 0 


call 

printf 

.text:004 01155 


add 

esp, 8 

.text = 004 0115 8 


jmp 

short loc 401195 

.text:004 0115A 

loc 

40115A: 

; CODE XREF: SecondCheck+142 

.text:004 0115A 


push 

offset aExit 0 ; "Exit" 

.text:004 0115F 


lea 

eax, [ebp+Dst] 

.text:004 01165 


push 

eax ; Strl 

.text:004 01166 


call 

strcmp 

.text:004 0116B 


add 

esp, 8 

.text:004 0116E 


test 

eax, eax 

.text = 004 0117 0 


j nz 

short loc 401188 

.text = 004 0117 2 


lea 

ecx, [ebp+Dst] 

.text = 004 0117 8 


push 

ecx 

.text = 004 0117 9 


push 

offset aOperationSCo 3 ; "Operation: %s: CompletedV 

.text:004 0117E 


call 

printf 

.text:004 01183 


add 

esp, 8 

.text:004 01186 


jmp 

short loc 401195 

.text:004 01188 

loc 

401188: 

; CODE XREF: SecondCheck+170 

.text:004 01188 


push 

offset alnvalidCommand ; "Invalid command failure. 

Please try aga' 




.text:004 0118D 


call 

printf 

.text = 004 01192 


add 

esp, 4 

.text = 004 01195 

loc 

401195: 

; CODE XREF: SecondCheck+C8, SecondCheck+F9 

.text:004 01195 


push 

80h ; Size 

.text:004 0119A 


push 

0 ; Val 

.text:004 0119C 


lea 

edx, [ebp+Dst] 

.text:004 011A2 


push 

edx ; Dst 

.text:004 011A3 


call 

memset 

.text:004 011A8 


add 

esp, OCh 

.text:004 011AB 


jmp 

loc 40104C 

.text:004 011B0 

loc 

4011B0: 

; CODE XREF: SecondCheck+53 

.text:004 011B0 


mov 

al, 1 
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text:004011B2 

pop 

edi 

text:004011B3 

pop 

esi 

text:004011B4 

mov 

esp, ebp 

text:004011B6 

pop 

ebp 

text:004011B7 

retn 


text:004011B7 

SecondCheck 

endp 


As you can see, this function is larger than all the others. Although our techniques so far 
are promising, we are moving into a more complicated function and, as such, need to shift 
our method of analysis a bit. When diving into a function a bit more complex, it is better to 
get an overall picture of the function calls made and different conditionals that may exist 
prior to actually getting down and dirty with its operations. On that note, let’s begin by 
mapping out the series of function calls made within this function. Specifically, we want to 
go into the graph view and see what IDA Pro has determined the layout of conditionals to 
be for us, allowing us to dissect much more information about the call structure between 
these functions, as shown in Table 4.2. 


Table 4.2 Functions 


memset() 

Fills both stack buffers within the function 
with nulls (0x00) 

printfO 

Outputs command request header text 

scanf() 

Receives and stores user input in the first 
buffer 

strncatO 

Copies the received data from the first buffer 
to the second 

strcmpO 

Compares the input provided by the user 
against a static command string 

printfO 

Outputs the appropriate command result 


As you can see, the overall structure of the function isn’t as complicated as it seems, except 
for the multiple conditional statements that exist within it. However, from a glance at the graph 
view, reading the output that print/ is specified to give on different conditions, it becomes obvi¬ 
ous that this method is some form of a command parsing engine; it takes commands with scan/ 
parses to check for them, and then outputs the appropriate results. Additionally, when looking 
at this in greater detail, we can see by the graph view that this is in fact an infinite loop as well. 
Although we would be able to see this if we analyzed the jump statements within this binary 
long enough, IDA Pro provides us with this information by showing an extra overall wrapping 
connection from the final step of the function to the beginning. It becomes obvious that this 
is, in fact, a simple parsing loop for command strings. (See Figure 4.2.) 
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Figure 4.2 Reading printf Output in Graph View 
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Solutions Fast Track 

Understanding Execution Flow 

0 IDA has many tools and views, such as the graph view, which assist in rapidly 
assessing the actual operations within a binary 

0 Always review the overall flow of a function or set of functions prior to diving into 
complete dissections of the entire block of code. 

0 It will help in the long run to obtain a level of comfort with other number sets; 
specifically with base-16 hex. The faster you are able to determine a value in your 
head, the less time you will spend with calculator open. 

0 Compilers will do funny things to binaries during compile time; these can 

sometimes be convoluted or pointless, but the differences are generally minimal in 
most optimization cases. 
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Recovering Hard Coded Password 

0 Identifying major conditional statements, such as final true and false results of 
comparisons is useful in many more settings than just hard coded passwords. 
However, beware of optimizers and obfuscation techniques which abuse jumps and 
comparisons in order to make the executable hard to read. 

0 Finding the root chunk of code which controls a true/false statement is a good 
critical step in identifying how to recover a password or bypass a major check 
within a binary. 

0 Developers and compilers alike do this in fickle manners sometimes for the sack of 
performance. Always be vigilant with what appears to be malignant portions of 
code; you never know what they might be for. 

Frequently Asked Questions 

Ql Why does IDA Pro have difficulty with identifying function arguments? 

A: Statically analyzing code without context is a difficult task to perform. With typecasting 
and optimizing compilers, this problem is compounded. It will always be safer to check 
the callers of routines rather than rely on IDA Pro alone in identifying what exactly the 
arguments to a function are. Even so, a vigilant eye needs to be kept open for common 
compiler code which manipulates structures because disassembly they are just groups of 
values. 
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Chapter 5 


Debugging 


Solutions in this chapter: 

■ Debugging Basics 

■ Debugging in IDA Pro 

■ Use of Debugging while Reverse Engineering 

■ Heap and Stack Access and Modification 

■ Other Debuggers 


0 Summary 
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Introduction 

Debugging is the act of locating bugs in software. Generally this is done by developers as 
bugs are worked out of their software. Debugging can take many forms. Beginning programmers 
often use output as a rudimentary form of debugging. This output can be printf statements in 
the case of C. 

The most popular story on the use of debugging involves actual bugs. The origin of the 
use of ibugi to refer to a programming mistake is attributed to Admiral Grace Murray Hopper. 
A moth got caught in one of the relays from the Harvard Universityis Mark II computer. The 
removal of the moth was coined debugging. 

Debuggers are programs themselves that run and monitor the execution of other programs. 
The debugger can control and alter the execution of the target program. Memory and variables 
can be monitored and altered as well. 


Debugging Basics 

Debuggers are an essential tool in the reverse engineeris toolbox. The ability to perform runtime 
analysis speeds up program understanding and reverse engineering. Certain tasks are easier 
within a debugger. Call chains can be watched instead of guessed. 

Tracking indirect calls is much easier during debugging. A call through a register is an 
example of an indirect call. IDA Prois static analysis tracks indirect calls in a very limited 
fashion. Cross references are not created. 

Debugging allows us to watch, observe, and guide our reverse engineering. We do not 
want to reverse engineer entire programs, but rather the interesting parts. 



User mode vs. kernel mode debuggers 

User mode debuggers operate on processes. They themselves are standard processes 
and as such are limited to what memory can be accessed. User mode debuggers cannot 
access kernel memory and thus are not useful to debug code operating in kernel 
mode. Kernel mode code can be the operating system, modules, and drivers. 

Attacks on driver vulnerabilities are becoming more popular. An attack at the 
kernel level bypasses many modern protections. An example is the Broadcom Wireless 
Driver Probe Response SSID Overflow (CVE-2006-5882). A specially crafted probe 
allows arbitrary code execution in the kernel. The attack is at a level lower than the 
firewall. The packets never get processed by the firewall. 
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Rootkits have been developed as drivers for years. Some malware has adopted 
the rootkit strategies, embedding themselves in the kernel. DRM software often has 
drivers that contain vulnerabilities as evidenced by CVE-2007-5587, which was discovered 
being exploited in the wild. 

Examples of user mode debuggers are IDA Pro and Ollydbg. 

Under the Windows environment the de facto kernel mode debugger was 
Compuware's SoftICE included in DriverStudio. SoftICE has reached end of life and is 
no longer supported. Fortunately Microsoft has been making great strides with its 
debugging tools. 


Breakpoints 

Breakpoints stop execution of a program within the debugger at a location of our choosing. 
Execution is stopped and control is passed to the debugger. Breakpoints come in two different 
forms: hardware and software. Hardware breakpoints, as their name indicates, require specialized 
hardware support from the CPU. 

Hardware Breakpoints 

The IA-32 family of processors provides support for four hardware breakpoints. The hardware 
breakpoints use special debug registers. These registers contain the breakpoint addresses 
as well as control information and breakpoint type. 

Breakpoint addresses are stored in debug registers DO to D3. In order to set breakpoints 
a size Yeld is needed. The possible sizes are 1, 2, or 4 bytes. Breaks on execution use a size 
of 1 byte. The possible sizes have been expanded to include 8 bytes for 64-bit CPUs. There 
are various conditions to trigger the breakpoints. 

■ Break on execution 

■ Break on memory access (reads and writes) 

■ Break on memory write only 

■ Break on 1/O port access (rarely used, most debuggers do not have this as an option) 

Software Breakpoints 

Software breakpoints can only break on execution. A software breakpoint is simulated because 
of the lack of hardware support. A software breakpoint replaces the original instruction with 
an instruction that traps the debugger. In IA-32 processors, the new instruction is generally 
INT 3 (OxCC).The debugger must keep track of the original instruction. 

When a software breakpoint executes, the INT 3 instruction passes control to the debugger. 
The debugger looks up the breakpoint in an internal table and replaces the INT 3 with the 
original instruction. The debugger then sets the instruction pointer back, making the saved 
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instruction the next instruction to execute. The entire process is not visible to the user; the 
debugger will display disassembly with the original instructions in place. 

Using Breakpoints 

Software breakpoints are used by debuggers more than hardware breakpoints. The main 
reason is that there is not a set limit to software breakpoints. Some anti-debug techniques 
involve calculating checksums on code sections to determine if any instructions are changed. 
Anti-debug techniques are covered in detail in Chapter 6. 

Hardware breakpoints can be set on memory, unlike software breakpoints. Breaking on 
memory access can allow us to look for use of tables or memory corruption. 

Single Stepping 

Single stepping is the process of executing a single instruction and then returning control to 
the debugger. The IA32 family of processors supports single stepping directly in the hardware. 
By executing a single instruction at a time, we have the ability to carefully monitor certain 
sections of code. However, it is impractical to debug an entire program using this method. 
Generally single stepping is used to understand select portions of code. 

From the CPU perspective, the debugger sets the TF (Trap Flag) on the EFTAGS register. 
Upon the execution of an instruction a debug exception will be generated. This debug 
exception is caught by the debugger as an interrupt, INT 0x01. 


Note 

Most debuggers provide step commands. Generally they are called step into 
and step over. From a user point of view the only differences happen on certain 
instructions, namely call and rep. 

When a call is encountered, a step into command will follow the call, while 
a step over will break on the instruction following the call. This is generally 
done by setting a breakpoint, not by single stepping till the return. 


Watches 

We need to keep track of variables. In source level debugging, variables are abstract named 
locations with values. Within assembly, variables are usually memory locations. The compiler 
can sometimes optimize a variable into a register. 

Watches are a way to display variables or useful expressions. They are updated whenever 
control is passed to a debugger, such as a breakpoint or single stepping. A watch can be a 
simple variable such as loop_counter or an expression like packet [offset *4]. 
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Exceptions 

Exceptions are used by programmers to catch errors.The following pseudo demonstrates an 
exception: 

_try 


} 


open (file) 


except 


printerror 

} 

The debugger can either stop on an exception or pass it on to the application. An exception 
does not necessarily mean something went wrong. Many times programs use custom exceptions. 
Custom exceptions are also a common anti-debugging technique. 

In Windows the exception 0xc000000005 is an Access Violation. This means that process 
attempted to access an address that is not mapped.You may have seen advisories that show: 

Exception C0000005 (ACCESS_VIOLATION reading [41414141]) 

The address 0x41414141 is not mapped to the process and is most likely part of an overwrite 
using As. We want the debugger to stop on access violations. 

Tracing 

Tracing is the process of executing a program and recording information along the way. The 
UNIX command strace runs an executable while intercepting all system calls including passed 
arguments. 

user@redbull:~$ strace Is 

execve("/bin/ls", ["Is"], [/* 31 vars */]) = 0 

brk(0) = 0x805c000 

access ("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) 
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 

0xb7eec000 

access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) 
open("/etc/ld.so.cache", 0_RD0NLY) = 3 

Tracing can be performed within a debugger, recording various levels of detail. 
Instruction level tracing is the most detailed. After executing each instruction, register values 
are recorded. This level of detail is not needed and the process is very slow. 

Function tracing can set breakpoints at the entry of all functions or single step till a call. 
When a function is called, the execution is stopped. The debugger will record the arguments 
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to the function and optionally can record data such as registers and the caller. Execution 
then resumes. 

While function tracing is much faster than instruction tracing, it sometimes does not 
provide the necessary detail. Ideally we want to trace basic blocks. Basic blocks are sequential 
instructions that are executed without taking a branch. This type of tracing helps determine 
why certain branches are taken and, if needed, how to modify input in order to take different 
branches. While basic block tracing is much faster than instruction tracing, it is still slow. 
Starting with the P6 line of processors, Intel included hardware support to trace branches. Newer 
processors have more functionality in this area, but they all use MSR registers. 

Intel has documented Last Branch Recording in Chapter 18 of the Intel ® 64 and IA-32 
Architectures System Programming Guide. New research in this area has been published and 
proof of concept tools have been released. See www openrce. org/blog/view/535/Branch_ 
Tracing_with_Intel_MSR_Registers. 

Debugging in IDA Pro 

IDA Pro comes with a built-in debugger, which was introduced in version 4.50. The debugger 
is implemented as a plug-in. This is a true testament to the extensibility of IDA Pro. 

IDA isnit limited to debugging the local system. The debugger can operate locally as well 
as remotely over the network. The debugging clients allow IDA to debug other machines 
running different operating systems and even CPUs. Authentication is available, but best practices 
suggest debugging only over the local network. 

IDA Pro supports the following debugging environments: 

■ Win32 Local 

■ Win32 Remote 

■ Win64 Remote 

■ Linux Remote (x86 only) 

■ OSX Remote (x86 only) 

■ WinCE Remote (ARM only) 

Note 

IDA Pro can change register values from the GUI. However, memory locations 

cannot be changed from the GUI. 

IDC, IDA Pro's scripting language, must be used to change memory locations. 

Memory locations include data and executable code. The IDC functions such 

as PatchByteQ, PatchWordQ, and PatchDwordQ must be used. 
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The debugger menu option is made available if the binary being analyzed matches one 
of the previous targets listed. Debugger settings are available from the debugger menu option 
as shown in Figure 5.1. 


Figure 5.1 Debugger Setup Options 


Debugger setup 

r Events 

□ Stop on debugging start 

□ Stop on process entry point 

□ Stop on thread start/exit 
n Stop on library load/unload 
l~~] Stop on debugging message 

rLog— 

0 Segment modifications 
0 Thread start/exit 
0 Library load/unload 
0 Breakpoint 
0 Debugging message 

Options 

0 Reconstruct the stack 

f~l Show debugger breakpoint instructions 

0 Set as just-in-time debugger 


□0 


EXCEPTIONS: 


Edit 


Reload 


OK 

Cancel 


Help 


Some of the notable options are: 

■ Stop on debugging start n This option will stop before the entry point in the case 
of a PE binary with a TLS section. 

■ Stop on entry point n This option stops at the listed entry point. Some initialization 
may have been completed. 

■ Set as just-in-time debugger n Windows allows a debugger to be set as default 
when programs crash. 

■ Exceptions n This option controls how IDA handles exceptions, whether they are 
passed to the application or not. 
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Tools & Traps... 


Writing your own IDA Pro debugger client 

The IDA SDK allows you to write plug-ins. Included in the SDK is the source code to the 
Linux Debugger plug-in/client. The plug-in's source code shows the interface to work 
with IDA. 

Using the provided source code a plug-in/client could be written for an operating 
system or architecture not currently supported. 


Use of Debugging 
while Reverse Engineering 

In order to demonstrate debugging within IDA Pro we will use Netcat as an example. Netcat 
is a network tool, whose name comes from the combination of network and the UNIX 
command cat. You can pipe data to and from other programs over a network, which is why 
it is known as the iTCP/IP Swiss army knife! 

In December 2004, vulnerability was reported in Netcat for Windows 1.1 (www.vuln- 
watch.org/netcat/netcat-lll.txt).We will be analyzing the vulnerable version 1.1 (http:// 
packetstormsecurity.org/UNIX/netcat/ncllnt.zip), while the patched version is 1.11 (www. 
vulnwatch.org/netcat/). 


Note 

Netcat is a networking tool capable of allowing remote access. Some antivirus 
vendors classify it as a hacking tool. Please be aware of this and take the 
necessary precautions. 


The vendor advisory describes a remote buffer overEbw when using the i-ei option. 
The vulnerability resides in the SessionWriteShellThreadFn function within the dosexec.c 
source Me. Select parts of the vulnerable function are shown in the following code snippet: 

static VOID 

SessionWriteShellThreadFn(LPVOID Parameter) 
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PSESSION_DATA Session = Parameter; 

BYTE RecvBuffer[1] ; 

BYTE Buffer[BUFFER_SIZE] ; 

BYTE EchoBuffer[5] ; 

DWORD BytesWritten; 

DWORD BufferCnt, EchoCnt; 

DWORD TossCnt = 0; 

BOOL PrevWasFF = FALSE; 

BufferCnt = 0; 

//Loop, reading one byte at a time from the socket. 

while (recv(Session->ClientSocket, RecvBuffer, sizeof(RecvBuffer) , 0) != 0) 

{ 

EchoCnt = 0; 

Buffer[BufferCnt++] = EchoBuffer[EchoCnt++] = RecvBuffer[0]; 
if (RecvBuffer[0] == '\r') 

Buffer[BufferCnt++] = EchoBuffer[EchoCnt++] = '\n'; 

//Trap exit as it causes problems 
if (strnicmp(Buffer, "exit\r\n", 6) == 0) 

ExitThread(0) ; 

// 

//If we got a CR, it's time to send what we've buffered up down to the 
//shell process. 

if (RecvBuffer[0] == '\n' || RecvBuffer[0] == '\r') { 

if (! WriteFile(Session->WritePipeHandle, Buffer, BufferCnt, 
SBytesWritten, NULL)) 


break; 

} 

BufferCnt = 0; 

} 

} 

ExitThread(0) ; 

} 

The function contains a receive loop with two possible exits. The Yrst exit occurs if the 
buffer contains an exit\r\n string. The second exit requires either a \n or \r as the received 
byte and a failure for the WriteFile call. 
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Note 

Netcat sends a \n as a newline when run from Windows. The command to 
terminate is exit\r\n. In order to terminate the newline needs to be sent as \r\n. 


After opening nc.exe in IDA Pro, the debugger tab is available. If an executable format is 
supported by one of the debuggers, the debugger tab will be visible. 

Process options need to be conYgured. SpeciYcally, we need to conYgure the command 
line arguments. The vulnerability is only present when the -e option is used. In order to use 
this option we must also supply the -/ (listening) options as well as the -p (port number option) 
The -e option executes a program passing any input it receives over the network. We donit 
need the executed program to do anything, so we can use the more program. Figure 5.2 
shows typical process options using our command line arguments. 


Figure 5.2 Debugger Application Setup 


Debug application setup 







Application DAreverseVicI 1nt\nc.exe 

_U 

input file 

D: \reverse\nc11 nt\nc. exe 

a 

Directory 

D:\reverse\nc11nt 

v! 

Parameters 

-1 -p 2323 -e more 

3 


Hostname | v j Port 1 23946 v | 

Password J 

I I Save the network settings as default 


OK 

Cancel 


Help 


Note 

If we are debugging a dll, the setup is slightly different. The dll would be in 
the input file box, while the application that uses the dll would go in the 
application box. 

This type of setup is very common with Internet Explorer. The iexplore.exe 
binary does very little work and leaves all the heavy lifting to dlls. 
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Debugger hotkeys are: 

■ F9 Start debugger/Continue process (if already debugging) 

■ F2 Set/Remove breakpoint 

■ F7 Step into 

■ F8 Step over 

■ CRTL + F7 Run until return 

■ F4 Run to cursor 

■ CTRL + F2 Terminate process 

We Ynd the Session WriteShellThreadFn function by looking at the imports for WriteFile. 
From the cross references we determine that the function address is .text:00401520.We then 
set a breakpoint at the beginning of the function. Figure 5.3 shows the graph view of the 
function. The function has been renamed to SessionWriteShellTItreadFn and stack variable buf 
has been renamed to RecvBuffer for readability. 

Figure 5.3 SessionWriteShellThreadFn Function 

HNut | 

; Attributes: noreturn 

; DWORD _stdcall SessionWriteShellThreadFn(LPUOID) 

SessionWriteShellThreadFn proc near 

RecuBuffer= byte ptr -OCDh 
NunberOfBytesWritten= dword ptr -OCCIi 


Buffer 

= byte 

ptr -0C8h 

arg_B= 

duo rd 

ptr 4 

SUb 



lea 

eax, 

[esp+0D0h»RecuBuffer] 

push 

ebp 


nou 

ebp. 

ds: iwprecu 

push 

esi 


push 

edi 


nou 

edi, 

[esp+0DCh*arg O] 

xor 

esi, 

esi 

push 

esi 

; flags 

push 

1 

; len 

nou 

ecx, 

[edi+OGh] 

push 

eax 

; buf 

push 

ecx 

; s 

call 

ebp 

imp rifcv 

test 

eax. 

eax 

jz 

short 

1ocJi015BC 


Push F9 and the debugger will start. The different windows will rearrange themselves. 
The debugger is running nc.exe using the passed arguments. Since there hasnit been a connection 
yet, our breakpoint hasnit been hit. 
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Start a cmd.exe shell and we will use another instance of Netcat to connect to the debugged 
one. This Netcat will be called the Netcat client in order to differentiate between the debugged 
Netcat. Use the command line: 


nc localhost 2323 

At this point we will hit our breakpoint. The register window will contain values similar 
to Figure 5.4. A stack window will be displayed similar to Figure 5.5. 

Figure 5.4 Registers 
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Figure 5.5 Stack 
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The register window shows the register values on the left and the right side contains any 
interpretation of the values. Registers can be changed by either right-clicking in the value 
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box or typing directly into the value box. New views are available by right-clicking on 
registers and most addresses. The views can be assembly or hex. 

Stepping with the F8 key (step over) will avoid going into the actual recv call. The recv 
call will block until it receives data. Type hello\n into the Netcat client. The recv call will 
return now that it has received data. We can continue single stepping. 

1. The basic block in Figure 5.6 does the following: 

2. Reads the single byte from the recv call into register al 

3. Writes the byte into the buffer 

4. Increments a buffer counter in register esi 

Figure 5.6 Reading and Writing Bytes to the Buffer 


BKNUA 

loc 401551: 

mou 

al, [esp*0f0h+RecoBuEfer] 

(HO u 

[esp+esir0E0h+Buffer], al 

inc 

esi 

crop 

al, • • ■ n 

i nz 

short loc_401564 


We can set another breakpoint on the instruction cmp al, OxOd in the basic block from 
Figure 5.6. Pressing F9 (continue) runs the program until another breakpoint is hit. Execution 
will stop at the breakpoint we just set. Notice the value of esi has incremented after each 
character. The character can still be seen in register al. Right-click the breakpoint and select 

disable breakpoint. 

After the entire command from the Netcat client has been copied into the buffer, WriteFile 
is called. Figure 5.7 shows the basic block containing the call. Set a breakpoint on test eax, 
eax, which is the instruction following the WriteFile call. When we continue (F9), the debugger 
will stop on this instruction. 


Figure 5.7 Basic Block Containing the Call 


H Ld 1 

loc 40158F: 


mou 

edx, 

[edi+4] 

lea 

eax. 

[esp+0E0h+NumberOfBytesWritten] 

push 

o 

; lpOoerlapped 

push 

eax 

; lpNumberOfBytesUritten 

lea 

ecx, 

[esp+0E8h+Buffer] 

push 

esi 

; nNumberOFBytesToWrite 

push 

ecx 

; lpBuFfer 

push 

edx 

; hFile 

call 

ebx 

WriteFile 

test 

eax. 

eax 

jz 

short 

loc 4015BB 
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We know the size of the buffer; it is 0xc8 bytes according to Figure 5.7. However, the 
stack looks different than expected since this function is called by CreateThread(). 

We can set a conditional breakpoint on the instruction cmp al, OxOd from the basic block 
shown in Figure 5.8. In order to set a conditional breakpoint, right-click on the disabled 
breakpoint and select Edit breakpoint. Enter esi == 0xc8 | | esi == Oxcc in the condition 
box as in Figure 5.8. The breakpoint will hit at the last location of the buffer and then upon 
overwriting the next DWORD. 


Figure 5.8 Conditional Breakpoint 


Breakpoint settings 


□a 


Address 0x0040155A 


0 Enabled 0 Hardware breakpoint 
Hardware breakpoint settings 
Size: T1 

Modes: 

Condition 

esi == 0xc8 II esi == Oxcc 


Actions 

0 Break 0 Trace 


OK 


Cancel 


Warning 

The condition box takes an IDC statement, which is evaluated. A common 
mistake is using a single equal = (assignment), when a double == (evaluation) 
is needed. This is a classic bug type in C code. 


We need to send more data. From the stack view, it appears that 272 bytes will write 
past the end of the page. Other machines or operating systems may have different memory 
layout. The easiest way to send the data is to build a string in a text editor and then paste it 
into the Netcat client. 
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Figure 5.9 Stack 
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After the breakpoint hits, the stack looks like Figure 5.9.There is still data to be read and 
there is very little space left on the page. The next breakpoint is the Yrst of the stack corruption. 
When the program is allowed to continue, we see the warning and then the exception will 
come up as shown in Figure 5.10. 

Figure 5.10 Exceptions 

Exception handling 

The execution wiW be resumed after the exception 
Do you want to pass the exception to the application? 

If you answer yes, the application's exception handler 
will be executed if there is one. 

The control of the application might be lost 

[ Change exception definition | 

[ Xes | | Mo 1 | Cancel | 
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The exception is conYgured to stop the program. We can go into Change exception 

definition and select Pass to application. See Figure 5.11. 


Figure 5.11 Handling Options 


Exception handling 

Code: 0xC0000005 

Name: EXCE PT10 N_ACCESS_VIOLAT ION 

r_ i 

M Stop program 
0 Pass to application 


□S 


OK 


Cancel 


Help 


The program will terminate with this exception. IDA Pro has recorded the exception in 
the log window: 

nc.exe:The instruction at 0x401555 referenced memory at 0xF50000.The memory could 
not be written (0x00401555 -> 00F50000) 

Debugger: Thread terminated: id=00001660 (exit code = 0xC0000005). 

Debugger: Thread terminated: id=000017AC (exit code = 0xC0000005). 

Debugger: Process terminated (exit code = C0000005h). 

Heap and Stack Access and Modification 

Detecting memory corruption is important to the reverse engineer. Stack and heap overEbws 
are attacks that overwrite and corrupt memory. 

The debugger can be used to detect memory corruption. Some ways of detecting corruption 
can be done manually while others are more applicable to being scripts or plug-ins. 

Microsoft began adding stack cookies to their compiler beginning with Visual Studio 
2003, using the GS command line switch. At the entry of a function, a stack cookie is placed 

on the stack. The cookie is calculated by taking a global security cookie,_security_cookie, 

and XORing it with the esp register. During an exit of the function, the stack cookie is 

XORed with the esp register. The result of the operation should be security_cookie. This value 

is passed to the_ security_check_cookie() function. If the passed value matches_ security_cookie, 

then_ security_check_cookie() returns allowing the original function to continue as designed. 

A more detailed explanation is available here: http://uninformed.org/index.cgi?v=7&a=2&p=l. 
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The idea of the protection is that the cookie check will fail if the stack has been corrupted. 
If the check fails, the process will be terminated with an exit code of 0xc0000409. In order to 

catch the stack corruption, we can set a breakpoint in security_check_cookie(), as shown in 

Figure 5.12. Alternately, the breakpoint can be set directly on the_ report_gsfailure() function. 

The security_check_cookie() function is compiled in statically and the address will change 

depending on the binary. 


Figure 5.12 Setting a Breakpoint 


IB N L»1 


_security_check_cookie proc near 

cmp ecx, securitycookie 

jnz_short loc_i>613F7_ 


b N m 


rep retn 


I B N ui 


loc 4013F7: 


lure 

_security_check_cookie endp 


Checking for heap corruption is more dependent on the operating system being used. 
Windows XP SP2, Windows 2003, and Vista have various methods of heap protection built 
in. However, unlike the GS stack protection, the heap protections are part of the operating 
system.Various techniques have been developed to pass heap protections, but data from a 
fuzzer will most likely be caught by these protections. 

There are multiple checks and simple breakpoints may not be sufYcient. Debugger based 
scripting or a plug-in would be ideal. The heap functions can be hooked to provide allocation 
data. The protection functions can be hooked to report corruption. For systems without such 
thorough protection functions, hooked functions containing checks could be added. Rather 
than stopping attacks, the checks notify us of corruption as soon as possible. 


Warning 

Debuggers can change the environment and behavior of a process. 

Processes started from a debugger use a debug heap, unlike starting the 
process normally. Attaching to a process is not affected. This difference is 
important when looking for heap corruption. In order to disable the use of 
the debug heap, set the environment variable _NO_DEBUG_HEAP to 1. 

set NO DEBUG HEAP=1 
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Microsoft's gflags.exe utility allows the setting of many debugging options. 
gflags.exe is part of Debugging Tools for Windows (www.microsoft.com/whdc/ 
devtools/debugging/default.mspx). 

Operating system code can have debugger checks. kernel32.Unhandled 
Exception Filter alters its behavior based on the presence of a debugger. This 
behavior was originally mentioned in Dave Aitel's paper "MSRPC Heap Overflow - 
Part II" and subsequently in the Shellcoder's Handbook. 


Other Debuggers 

The debuggers within IDA Pro are very usefiil.You have full access to the static analysis, renamed 
functions, and other parts of code that have been reverse engineered. Like any type of tool, 
people have preferences for different tools. There are different debuggers available to the 
reverse engineer. 

Each debugger has advantages and disadvantages. It usually comes down to a matter of personal 
preference. The following paragraphs provide a brief overview of some other debuggers. 

Windbg 

Debugging Tools for Windows is a collection of debuggers from Microsoft (www.microsoft.com/ 
whdc/devtools/debugging/default.mspx). There are two different versions available from 
Microsoft, a 32-bit and 64-bit version. 

Debugging Tools for Windows 32-bit Version runs on: 

■ Windows NT 4.0 

■ Windows 2000 

■ Windows XP (32-bit or 64-bit) 

■ Microsoft Windows Server 2003 (32-bit or 64-bit) 

■ Windows Vista (32-bit or 64-bit) 

■ Windows Server 2008 (32-bit or 64-bit) 

Debugging Tools for Windows 64-bit Version runs on: 

■ Windows XP (64-bit) 

■ Microsoft Windows Server 2003 (64-bit) 

■ Windows Vista (64-bit) 

■ Windows Server 2008 (64-bit) 

The 64-bit version of the Debugging Tools for Windows should only be used if debugging 
native 64-bit applications. 
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Windbg is the debugger you will most likely use, although other debuggers are included, 
such as NTSD, CDB, and KD. Windbg is a user and kernel mode debugger. One of its 
primary beneYts is the tight integration with Windows. 

Ollydbg 

Ollydbg is a free Win32 user mode debugger, available from www.ollydbg.be. Although source 
code isnit available, there is an SDK provided. Many plug-ins have been written for Ollydbg. 

Ollydbg is a very popular debugger among reverse engineers. It was written from the 
reverse engineer standpoint. Some notable plug-ins include scripting, anti-anti-debugging, 
and tracing. There are many tutorials available for Ollydbg ranging from basic reversing to 
security bug hunting to breaking software protections. 

The current version is 1.10 and is no longer supported as the author is working on the 
upcoming 2.0 release full time. There have been various vulnerabilities reported in the 
debugger, including a format string vulnerability (CVE-2004-0733). Packers use these 
vulnerabilities in order to prevent debugging. However, reversers have released plug-ins which 
patch these vulnerabilities, one of the most popular being Oily Advanced (www.openrce.org/ 
downloads/details/241 / 011y_Advanced). 

Immunity Debugger (Immdbg) 

Immdbg is a free debugger released by Immunity Inc. (www.immunityinc.com/products- 
immdbg.shtml). When you Yrst run Immdbg, you will notice that it is ad supported. These 
are not ads for the next hot stock, but rather they are ads purchased by companies looking 
for security talent. If Immdbg looks similar to Ollydbg, it is not by mistake. Immunity Inc. 
licensed the source code to Ollydbg in order to add features useful for exploit development. 

Having a source license allows them to Yx bugs. New features include graphing, a command 
line, and remote debugging. The standout new feature is the built-in Python scripting. Some 
sample scripts are included to demonstrate the Python API. Other scripts have been released 
by users on the Immdbg forum. 

PaiMei/PyDbg 

PaiMei is a reverse engineering framework (http://paimei.openrce.org/). It is written in 
Python and has scripts to use analysis from IDA Pro. One of PaiMeus key components is PyDbg. 
PyDbg is a scriptable debugger written in Python allowing it to integrate with IDAPython 
(http://d-dome.net/idapython/). 

IDAPython is an IDA Pro plug-in that allows scripting. It wraps many of the IDC and 
SDK functions. Unfortunately IDA Python does not wrap many of the debugger calls. 
However, IDAPython can use PyDbg in order to debug and combine runtime analysis with 
IDA Prois static analysis. 
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GDB 

All the other debuggers discussed thus far have been for Windows operating systems. The 
GNU Project Debugger (GDB) is available for most UNIX systems. GDB is primarily a 
source level debugger. However, GDB can also operate at the assembly level. 

GDB uses a text-based interface, although numerous graphical front ends have been 
developed.They communicate with GDB using MI (Machine Interface). Scripting languages 
can drive GDB by using MI. 



Packers 

What is a packer? Packers are most commonly used in Win32 environments. They compress 
executables and uncompress the image in memory when executed. Thus, in order to 
analyze the binary, the actual uncompressed image is needed. A common open source 
packer is UPX, http:///upx.sourceforge.net. UPX is designed to operate in both directions; 
it is able to restore a packed binary back to the original binary. Most packers are not 
designed this way and there are modifications often made to UPX to prevent the 
symmetric behavior. 

Often the binary is run within a debugger or emulated environment until the 
original entry point (OEP) is determined. Memory is then dumped along with an 
appropriate PE header. Generally the imports are destroyed, so the imports are added 
back in. This is the basic method for extracting the original image. There are variants 
as the packing/unpacking arms race continues. 

Packers are used in many other programs besides malware. The goal of the 
packer is to make reverse engineering more difficult, while also lowering the file foot¬ 
print. Some of the anti-debugging/reversing techniques will be discussed in Chapter 6. 
Software protection makes use of packers. This can include software from shareware 
to commercial packages. Any software dealing with DRM will also typically use packers 
and/or anti-debugging. Skype, a popular telephony program, makes substantial use of 
these techniques. In order to evaluate or binary audit packed software for vulnerabilities, 
the unpacked image is needed. 
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Summary 

IDA Prois debugger is very powerful and allows for much greater program understanding 
than static analysis alone. The debugger can operate locally as well as remotely with the most 
common operating systems. 

A beneYt to using the IDA Pro debugger over other debuggers is the availability of any 
reverse engineering work we have done. This includes renaming functions, tables, and local 
variables. 

There are times when IDA Prois debugger is not the best solution.Various other 
debuggers are available. 


www.syngress.com 


This page intentionally left blank 



Chapter 6 


Anti-Reversing 


Solutions in this chapter: 

■ Debugging 

■ Example Overview 

■ Obfuscation 


0 Summary 


109 






110 Chapter 6 • Anti-Reversing 


Introduction 

Anti-debugging is a natural occurrence that should be expected; as soon as people started 
reversing applications it was only a matter of time before other people started trying to 
make it harder, implausible or even impossible for someone to reverse their application. 
Anti-debugging, like reverse engineering or coding in assembly, is an art form. The trick 
of course is to try to stop the person reversing the application. However, in most instances 
these attempts range from the absurdly lame to the truly difficult. At first it may be your 
presumption that only malicious software would seek to impede your reversing progress, 
but really you will find it everywhere and indeed there are legitimate jobs out there just 
for people to create such anti-reversing technologies, especially in the video game industry. 
In this chapter, what we hope to do is write a fairly comprehensive overview of anti¬ 
debugging and anti-disassembly techniques. Make no mistake—these tricks and techniques 
are designed to make your life and job more difficult, and in some instances there really is 
no good solution to getting around the problem presented. However, it is important to 
know and remember one thing: Given enough time and motivation, the reverse engineer 
always wins. 

First, if we really want to understand anti-reversing, then a little knowledge of exactly how 
debugging and disassembling is done would be helpful. This of course is not meant to be an 
all-encompassing perspective on the art, but rather a brief introduction to how it works, with 
the intent of using that knowledge as leverage to understanding anti-reversing techniques. 

Debugging 

To really understand debugging, a brief tour of the various interrupts and debug registers is 
necessary, especially in regards to what state changes occur in the process. The Intel Software 
Developers Manual is once again the best place for reference in this regards, as it goes much 
deeper into details than I can. However, basically the IA-32 platform handles debugging 
through one of a couple of means. 

First, the debug registers. There are eight debug registers supporting the ability to monitor 
up to four addresses. The registers themselves are accessed through variants of the MOV instruction 
with the debug registers potentially serving as either the destination or source operands. It 
should be known that accessing the registers is a privileged process requiring ring-0 privileges, 
which of course is a limiting feature but considering the power they give it makes sense. For 
each breakpoint, it is necessary to specify the address in question, the length of the location 
(ranging between a byte and a dword), a handler when a debug exception is generated and 
finally whether this breakpoint is even enabled. The first three debug registers, DRO through 
DR3 can contain three 32-bit addresses that define the address where breakpoints should 
occur. The next two debug registers, DR4 and DR5 respectively, have alternating roles depending 
on mode of operation. When the debugging extensions (DE) flag is set in control register 4 (CR4), 
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DR4 and DR5 are reserved and cause an invalid-opcode exception when an attempt to 
reference them is made. If the flag is unset, then DR4 and DR5 are instead aliases for debug 
registers 6 and 7. 

Debug register 6 (DR6), is also known as the debug status register and indicates the 
results of conditional checks at the time of the last debug exception. DR6 is accessed as a 
bit pattern, with bits zero through three being related to the first three debug registers. Each 
of these bits indicates which breakpoint condition was met and caused a debug exception to 
be generated. Bit 13 of DR6 when set indicates that the next instruction references a debug 
register and is used in conjunction with a portion of DR7, which we will describe momentarily. 
Bit 14 is perhaps the most interesting for our purposes; it indicates when set that the processor 
is in single-step mode, which is yet another concept we will introduce momentarily. Finally 
in use is bit 15, which indicates that a debug exception was raised as a result of a task switch 
when the debug trap flag was set. Finally, we arrive at debug register 7, or DR7 as you 
might have guessed. This is a very interesting register to hackers of all kinds as it’s also known 
as the debug control register and like DR6 is interpreted as a bit field. The first byte of this 
register corresponds to whether a breakpoint is active, and if so its scope. Bits zero, two, 
four and six determine whether a debug register is enabled or not, with bits one, three, five 
and seven corresponding to the same breakpoints but on a global scope. The scope in this 
instance is defined as whether the breakpoint persists through task switches, with globally 
enabled breakpoints being available to all tasks. In later versions of the processor, according 
to the Intel manual, bits eight and nine are not supported. However traditionally they provide 
the ability to determine the exact instruction that caused the breakpoint event. Next we 
have bit 13; this is an interesting bit as it allows for breaking before accesses to the debug 
registers themselves. Finally we have bits 16 through 31. These bits determine what types of 
access cause a breakpoint, and what the length of the data at the address is. When the DE 
flag in CR4 is set, bits 16 to 17, 20 to 21, 24 to 25 and 28 to 29 are interpreted in the 
following manner: 

00 - Break on execution 
01 - Break on write 

10 - Break on I/O read or writes 

11 - Break on read and writes but not instruction fetches 

However, when the DE flag is not set the interpretation remains the same except for 
values of 10 which are undefined. Bits 18 to 19, 22 to 23, 26 to 27 and 30 to 31 correspond 
to the lengths of the various breakpoints with a value of 00 indicating that the length is 1, 
and 01 indicating a 2-byte length. 10 is undefined on 32-bit platforms, with it indicating a 
length of 8 bytes on 64-bit processors. Finally, as you might have deduced, a value of 11 
indicates that the length in question is 4 bytes in length. 

Now it may seem a little confusing trying to tie all these bit sequences together with 
DR0 to DR3, but it really isn’t. Each 2-bit combination corresponds to a given sequential 
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register in the range of DRO through DR3.These lengths must be aligned on certain boundaries 
dependent on their size—for instance 16-bit values need to be on word boundaries and 
32-bit ones on double-word boundaries. This is enforced by the processor by masking the 
relevant low-order bits of the address; thus an unaligned address will not yield performance 
as expected. An exception is generated if any addresses in the range of the starting address 
plus its length are accessed, effectively allowing for unaligned breakpoints by using two 
breakpoints; each breakpoint is appropriately aligned and between the two of them they cover 
the length in question. Now one last note of interest here—when the breakpoint access type 
is execution only, then the length specified should be set to 00; any other value results in 
undefined behavior. 


Note 

Interestingly enough, the debug registers have not received tremendous 
amounts of attention publicly. However, privately there are numerous and 
quite effective rootkits and backdoors that make use of them. For instance, 
if so inclined, a person could hide a process in a linked list of processes by 
setting a global access breakpoint on the pointer to their process structure. 
When an access to that address occurs, a debug exception occurs and they 
can redirect into their handler and perform any number of tasks, including 
returning the address of the next process in the list. 

To make matters worse, they can enable the GD flag in DR7 and cause 
accesses to the debug registers themselves to have an exception generated, 
thwarting even attempts to inspect the registers to check for their current 
configuration. 


Now we’ve mentioned debug exceptions throughout the description of the registers, 
but we haven’t really done anything beyond mention them. The IA-32 processor has an 
interrupt vector specified in the interrupt descriptor table which was described previously in 
Chapter 2. Of these, the processor dedicates two interrupt vectors to these exceptions.These 
two interrupts are one and three, which are the debug and breakpoint exceptions, respectively. 
The debug exception, or INT 1, is generated by multiple events and DR6 and DR7 
should be consulted to determine what type of event occurred exactly. In the process of 
an exception there are two general classes, faults and traps. Essentially, the difference 
between the two classes is whether the instruction that generated the interrupt was executed 
or not by the time the handler gets control. In faults, control is handed to the handler first, 
whereas in traps execution control is passed to the handler after the instruction that caused 
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the exception is generated. Of these two classes, we have several different conditions that fall 
into the classes: instruction breakpoint, data and I/O breakpoint, general-detect, single-step 
and task-switch conditions. Of these the instruction breakpoint, general-detect and arguably 
task-switch conditions are fault class, while data and 1/O breakpoint and single-step conditions 
are trap class. 

Instruction breakpoint class conditions are the highest priority exceptions and occur 
when an instruction at an address referenced in DRO to DR3 is attempted to be executed. 
We say that these exceptions are the highest priority meaning that they receive service first; 
however, there are instances where these events may not even be triggered We’ll delve into that 
a bit later though. Now if you recall in the description of the EFLAGs register earlier, there 
was the resume flag in present. The problem is that it’s possible for a debug exception to be 
re-raised as a result of this exception being a trap-class exception. This is where the resume 
flag comes into play, as it prevents looping of the debug exception. We’ll cover this also shortly 
when we start to delve into specific methodology. The other fault class condition is the 
general-detect condition; this is raised when the relevant bit in DR7 is set, protecting access 
to the debug registers. 

Data and 1/O breakpoints are trap-class conditions and are caused by data accesses of 
addresses in DRO to DR3. Data accesses are essentially any condition that’s not an execution 
attempt. The IA-32 processor contains an interesting quirk in that trap-class events occur 
after the instruction that caused them was executed. For instance, suppose you set a write 
breakpoint on address X whose value is 0 and then the application modifies address X, 
setting its value to l.When the exception handler receives control, the value at address X 
will be 1, not O.The Intel manuals suggest that applications that want to be able to interact 
with the original value should save it at the time of the breakpoint, although doing so 
creates an interesting but off-topic synchronization issue. Even more, these breakpoints, 
like instruction breakpoints, are not always exact; for instance, repeated execution of 
certain SIMD instructions can cause the exceptions to be raised until the end of the second 
iteration. 

Single-step exceptions are also trap-class conditions, and are one of the more common 
conditions encountered when debugging. These conditions are caused when the trap flag in 
the EFLAGs register is set. Just like every other type of exception, single stepping comes with 
its own quirks. For instance, generally speaking the trap flag is not modified in the process 
of performing various tasks; however, certain instructions like software interrupts and INTO 
instructions do clear the trap flag. This effectively means that, in order to maintain control, a 
debugger has to emulate these instructions and cannot directly execute them, that is if they wish 
to continue inspiring single-step exceptions. Finally, task-switch exceptions occur after a task 
switch if the trap flag in the new tasks TSS is set; the exception is raised after the task-switch 
but prior to the first instruction in the new task. 

Now, in addition to INT 1, there is also the breakpoint exception or INT 3.The 
breakpoint exception is interesting in that it allows for extension of breakpoints past the 
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number supported by the debug registers. However, it requires modifying memory an 
Achilles heel that we will talk about exploiting later. For now, all we really need to know is 
that it exists. 

Upon hearing all of this, an inspired reader with a good imagination might already 
begin to see the conditions that could be checked for and ways that interruptions might be 
avoided. However, people don’t purchase nonfiction books to inspire their imagination but 
rather to learn facts, so we’ll cover some of the complications that can occur in the rest of 
the chapter. We will explore mostly ring-3 based implementations of circumventing both 
disassembly and debugging, but to spice things up some we will also touch on some ring-0 
based concepts. 

Example Overview 

Because this is a somewhat tricky subject which can be difficult to explain exactly, especially 
within the constraints of a single chapter, we’re going to take the following approach. I’ve 
written a simple application, a silly network RPC server and client. What we’re going to do 
is take this (the server side component) application and harden it to reversing a bit, or rather 
do as much as we can within the constraints. Thus, we will have two products: the original 
program and the one we’ve hardened. The idea here is that we’re going to reverse the anti- 
RCE process and, by making one ourselves and looking at the generated code, when you do 
eventually run across this sort of stuff, you’ll have some concept of what’s going on and how 
to get around it. The example codes we will be using for this chapter are available for down¬ 
load from the Syngress website. We will not be looking at the client side of the application, 
and it is there simply for you to experiment with, and for me to verify that everything is 
working as expected. 

So without any further ado, let’s take a brief look at the application prior to doing anything 
to it, so you have some idea of what it started off looking like. Figure 6.1 shows the relevant 
section of the control-flow graph that’s generated when you go to View > Graphs > Control 
Flow, or if you hit F12, the options hotkey. This isn’t the full graph, as it’s not necessary 
and trying to fit it into a page was problematic. Basically, I want you just to get an idea of what 
we’re looking at. However, I highly encourage you to download the source, compile it and 
look it over. 
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Figure 6.1 Control Flow Graph 


serverusepr; ''RpcServerUseProtseqO failured\n" 


loc_401886: 

mov esi . esp 

lea eax, [ebp-*-BindingVector] 

push eaw ; BindingVector 

call ds: RpcSer ver I nqB i nd i ngs 

anp esi, esp 

call unknown.1ibname.l; Microsoft VisualC 2-8/ne 

mov [ebp-*-var_4], eax 

anp [ebp*var_4], 0 

jz short loc.4Q18AF 


loc_4013AF: 
mov esi, esp 

push offset Annoteticn; "example rpc server" 

push 0 j UuidVectoc 

mov ecx, [ebp+BindingVector] 

pjsh ecM ; BindingVector 

mov edx, IfSpec 

push edx ; IfSpec 

cal1 ds :RpcEpRegisterA 

anp esi, esp 

call unknown.!iboame.l; Microsoft VisualC 2-8/net runtime 

mov [ebp-»-var_4], eax 

anp [ebp-»-var_4], 0 

iz short loc_4018E6 


004018A2: 

push 

offset aftpcst 

call 

sub_4017F0 

add 

esp, -4 


t eRpcepregisterF \ "RpcEpRegisterO failure\n*' 
317FO 


loc_4018E6: 

irov esi, esp 

push 0 ; MgrEpv 

push 0 $ MgrTypeUuid 

mov eaw. IfSpec 

push eax \ IfSpec 

call dsrRpcServerRegisterlf 
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As you can see, this is obviously an application dealing with RPC in some form, and 
when we take a closer look it becomes clear that it’s intended to be the server component, as 
one can tell just by the API calls. As you examine the code, take note that there are multiple 
string constants, that there is really no attempt to obfuscate what the application is doing and 
so on. Now, let’s change that! If you feel a little lost at this point, pull the code down and take 
a closer look at it to get a better feel for it, as this is a chapter about anti-reversing, not about 
coding an RPC server and I really can’t delve into the details much further. 

Obfuscation 

As we noted earlier, there is very little doubt about what is going on in this program, or at 
least what appears to be going on. (One should be careful about forming conclusions in regards 
to the application’s purpose with such a small preview; however, in this case, what we see is 
what we get.) Let’s start off with one of the simpler, although more effective, techniques and 
obfuscate the code a bit. 

These types of techniques are common and really are something you should just accept 
as the normal routine; sans any other types of security, they generally won’t pose much of a 
problem for you. What we’re actually trying to do here is raise the bar as to who can read the 
code.You see, a decent percentage of people calling themselves “reverse engineers” or working 
as “incident responders,” really aren’t. Some of them can do little more than extract readable 
strings from the binary, others just read API calls, while some will take a guess of intent based on 
data in the imports section. An often-used trick—especially when dealing with packed binaries— 
is to modify some aspect of the binary so that if someone tries to dump the image straight 
from memory it will be corrupted. 

Taking this all into account, let’s take a look at the newly modified main routine in Figure 6.2. 

Figure 6.2 Modified Main Routine 


iHHul 


, *1 tributes : bp-based frame 

; int _cdccl nain(int argc,const char »-*argu f const char -cnup) 

main proc near 


oar 

23b- duord 

ptr 

?3bh 

oar 

2 iN= duord 

ptr 

-21BII 

oar 

IF U= duord 

ptr 

-1F hh 

oar 

1F0- duor d 

ptr 

-IF Oh 

oar 

. 1EC= dword 

ptr 

-1ECh 

oar 

1E8= dword 

ptr 

-1E8h 

oar 

1E**= drvurd 

plr 

- 1E*di 

oar 

_1E 0= dword 

ptr 

IE Oh 

oar 

IDG- duord 

ptr 

-IDCh 

oar 

1 D 8 = dword 

ptr 

10811 

oar 

1Dii= duord 

ptr 

-IDbh 

oar 

1D0- dword 

ptr 

ID Oil 

oar 

ibb= dword 

ptr 

ibbli 

oar 

_10b= duord 

ptr 

-19*ih 

oar 

190- duord 

ptr 

190M 

oar 

~ 1 HG= dword 

ptr 

-mn 

oar 

16C= duord 

ptr 

16Ch 

oar 

160- duord 

ptr 

-168h 

oar 

_16*»= dword 

ptr 

-16*di 

oar 

16 0= dword 

ptr 

-16 Oh 

oar 

15C= dword 

plr 

-15CII 

oar 

.158= byte ptr 

158b 

oar 

IS**- dword 

ptr 

-15*ih 

oar 

i3b= dword 

ptr 

-I3*in 

oar 

130= dunrd 

ptr 

1.3 Ofi 

oar 

12C- dword 

ptr 

12Ch 
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Looking at just the entry point reveals that this new binary could be significantly more 
complex, just by the number of local variables now present. Of course, we are also looking at 
different views of the code, so this isn’t quite as obvious as when we reach the first sets of 
instructions (see Figure 6.3). 


Figure 6.3 First Instructions 


ve»*_u — uwui u pci uv«ii 

uar_8= dword ptr -8 
uar_4= dword ptr -4 
argc = dword ptr 8 
argu= dword ptr OCh 
enup= dword ptr lOh 
arg_34- dword ptr 3Ch 
arg_70= dword ptr 78h 

push ebp 

nou ebp, esp 

sub esp, 348h 

nou eax, dword_4 0l *000 

xor eax, ebp 

nou [ebp+uar_C], eax 

push ebx 

push esi 

push edi 

nou eax, ds:dword_4031l8 

nou [ebp+uar_6C], eax 

nou ecx, ds:dword_40311C 

nou [ebp+uar_68], ecx 

nou dx, ds:word_403120 

nou [ebp+uar_64], dx 

nou al, ds:byte_403122 

nou [ebp+uar_62], al 

nou ecx, ds:dword_403124 

nou [ebp+uar_C4], ecx 

nou edx, ds:dword_403128 

nou [ebp«-uar_C0], edx 

nou eax, ds:dword_40312C 

nou [ebp+uar BC], eax 

nou ecx, ds:dword_403130 

nou [ebp*uar_B8], ecx 

nou edx, ds:dword_403134 

nou [ebp*uar_B4], edx 

nou al, ds:byte_403138 

nou [ebp*uar_B0], al 

nou ecx, ds:dword_403l3C 

nou [ebp+uar_fl4], ecx 

nou edx, ds:dword_403l4O 

nou [ebp+uar_B0], edx 


The trend in change of the code continues as we note that, where once there was the 
start of RPC calls, we now have a long series of byte copies from the data segment onto the 
stack. You may note that the stack is declining while the data index is increasing; this is typical 
of things like string constants initializing a stack variable. We'll come back and look at this in 
a moment. However, before we delve into the changes I want you to be able to see and get a 
feel for the differences in the program, as seen in Figure 6.4. 
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Figure 6.4 Differences in the Program 

rep movsa 
rovsw 

rov edx, ds:dword_403304 

rov [ebp*var_24], edx 

rov eax, ds:dword_4O3308 

rov [ebp+var_20], eax 

rov ecx, ds:dword_40330C 

rov [ebp+var_1C], ecx 

rov edx, ds:dword_40331O 

rov [ebp+var_18], edx 

rov eax, ds:dword_403314 

rov [ebp*var_14], eax 

rov ecx, ds:dword_403318 

rov [ebp*var_10], ecx 

push offset _nain 

call sub_401550 

add esp, 4 

rov dword_4O4420, eax 

cnp dword_404420, O 

jnz short loc_40222f 

- ^ "1 


1 


0 N ui 1 

loc 40222f: 


mou 

edx, ds:'iJeep 


push 

edx 


call 

sub 4O1550 


add 

esp, 4 


nou 

dword 404424, 

eax 

cmp 

dword 404424, 

o 

jnz 

short loc_402256 | 




Continuing down the code, we finally get to the first “real” portion of code, which is a 
call to the function sub_40155. There are two calls to it, the first passing the address of the main 
function in as an argument, the second passing in the address of the kernel32 function Sleep(). 
Now that we have some idea of what we’re getting ourselves into, let’s take a bird’s-eye view 
of the code (Figure 6.5) and see how crazy the structure of the application might be. 
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Figure 6.5 Complicated Structure of Application 
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And here we have it—no matter how awkward the code initially appears, it exhibits fairly 
typical structures, still showing a basic Boolean logic to its overall composition. Most of the 
complexity appears to be towards the start, and then we see a simple series of if () {if () [...] 
else [...]} else [..] structures. Notice the first error branches to the right, whereas the second 
error branches off to the left, continuing on down to process termination. 

Great! Now that we have some idea of what we’re working with, let’s go back and get a 
better idea of how exactly it changed and see if we can figure out how this little puzzle fits 
together, starting with that long series of copies from the data section to the stack. Starting at 
the beginning of the data, we have address loc_403118, so let’s jump there and see what we 
can determine (Figure 6.6). 


Figure 6.6 address loc_403118 


.rdata:0O483118 
.rdata:0O4031lC 
.rdata:00403120 
.(-data: 00483122 
.rdata:0O403123 
-rdata:00483124 
.rdata:00403128 
-rdata:004O3l2C 
.rdata:00403138 
.rdata:0O403134 
-rdata:08403138 
.rdata:0O403139 
-rdata:0040313C 
.rdata: 00403140 
.rdata:00403144 
.rdata:00403148 
.rdata:004O314C 
.rdata:00403150 
.rdata:00403151 
.rdata:00403154 
.rdata:004O3158 
.rdata:0040315C 
.rdata:0O403160 
.rdata:00403162 
.rdata:00403163 


dword_403118 

dword_40311C 

word_403120 

byte_403122 

dword_403124 

dword_403128 

dword_48312C 

dword_403130 

dword_403134 

byte_403138 

dword_40313C 

dword_403140 

dword_403144 

dword_403148 

dword_40314C 

byte_403150 

dword_403154 
dword_403158 
dword_40315C 
uord_403160 
byte_403162 


dd 2D23190Eb 
dd 3B021856h 
dm 233Ch 
db 0 
align 4 
dd 0C23192Eh 
dd 38585E47h 
dd 38171822h 
dd 2B260F7Eb 
dd 1E561F43h 
db o 
align 4 
dd 0C23192Eh 
dd 38585E47h 
dd 2E080622b 
dd 3B27146Ch 
dd 2C401459h 
db 0 
align 4 
dd 1023192Eh 
dd 38497E52h 
dd 38103C39h 
dw 3C5Ch 
db 0 
align 4 


0818 

XREF 

D8T8 

XREF: 

08 T8 

XREF 

08 TO 

XREF 

08T8 

XREF 

0818 

XREF 

08T8 

XREF 

08T8 

XREF 

08T8 

XRFF 

0818 

XREF 

0818 

XREF 

08T8 

XREF 

08 T 8 

XREF 

D8T8 

XREF 

0BT8 

XREF 

0BT8 

XREF 

DBTB 

XREF 

DBT8 

XREF 

08T8 

XREF 

08T8 

XREF 

08T8 

XREF 


maii>*16tr 

main»1Efr 

nain+27tr 

nain»32tr 

nain«3fltr 

nain»46tr 

main*52lr 

niainrSDTr 

nain»69lr 

nain*75tr 

rtain*80fr 

nain*8CTr 

main»98tr 

nain*83tr 

nain»0rtr 

nain»BBtr 

nain»C6tr 

nain*02tr 

nain'DFtr 

nain*F9lr 

nain*F7Tr 


.rdata:00403164 

dword 

403164 

dd 

0C23192Eh 

; 08T8 

XREF 

jrtaln*103lr 

.1-8313:00403168 

dword 

403168 

dd 

38585E47h 

; D8T0 

XREF 

_main+10ETr 

.rdata :0040316c 

dword 

40316C 

dd 

38011022b 

; 08T8 

XREF 

_main»118Tr 

.rdata:00403170 

dword 

403170 

dd 

383O0E47h 

; 08T8 

XREF 

main*l26lr 

,rdata:00483174 

dword 

403174 

dd 

413342h 

: 0818 

XREF 

main*131 fr 


As we examine the data that initially is referenced in the main() function, it becomes obvious 
that they are not ASCII characters. However, they appear to be NULL terminated, further 
supporting the idea of string constant initialization of local variables. Furthermore, we can take 
note that the data has some repeating patterns, for instance, look at how many sequences start 
with the bytes 0x0C2319.This may be indicative of weak encryption. We really won’t be able 
to tell for sure until we get to a section of code that interacts with this data. We’ll shelve the 
idea for now, but we can speculate that the program once accomplished a great deal with RPC 
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and had many string constants, and now contains a bunch of random NULL terminated data 
that appears to at least have a repeating first three characters in most of the cases. 

Now, knowing that there is not a lot we can do at the present time about the apparently 
encrypted data, unless we’re willing to track down where the data gets used, we’ll move on 
to the next portion and let the order of operations occur sequentially Next up, we have the 
function sub_401550 (Figure 6.7). 


Figure 6.7 Function sub_401550 


IjHNul 


; Attributes: bp-based Frane 

sub_401550 proc near 

oar 28= 

dword ptr -28h 

uar 24= 

dword ptr -24h 

gar 20= 

dword ptr -20h 

gar 1C= 

dword ptr -ICh 

gar 18= 

dword ptr -18h 

gar 10= 

dword ptr -10h 

gar 8= 

dword ptr -8 

gar 4= 

dword ptr -4 

arg_0= 

dword ptr 8 

push 

ebp 

nou 

ebp, esp 

push 

OFFFFFFFEh 

push 

oFfset unk 403588 

push 

oFFset except Handlers 

nou 

eax, large fs:0 

push 

eax 

add 

esp, 0FFFFFFE4h 

push 

ebx 

push 

esi 

push 

edi 

nou 

eax, dword 4O400O 

xor 

[ebp+oar_8], eax 

xor 

eax, ebp 

push 

eax 

lea 

eax, [ebp»uar 10] 

nou 

large fs:0, eax 

nou 

[ebp*gar_18], esp 

cnp 

[ebp+arg 0], 0 

jnz 

short loc_401591 


T1 


HNul 

now eax, 1 
retn 


_ t 

HNul 


loc_401591: 

nou eax, [ebp+arg_0] 

now [ebp+uarIC], eax 

nou [ebp+uar_4], 0 



a n ui 


loc_40159E: 

cnp [ebp+uar_1C], 0 

17 1 nr JittlAAR 
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As we examine the entrance of sub_401550, we see a function that’s a bit more normal 
looking than main(). As you may recall, in the first part of main() this is invoked twice, once with 
an argument of the address of main and once with a pointer to Sleep(). The user portion of the 
code starts at the cmp instruction testing arg_0 against 0 towards the end of the first box. One 
thing to note in the compiler-generated code is that there is an exception handler setup. 

So the first real thing we see is that the argument is tested against 0, or rather the pointer 
is checked against NULL, if it is, we branch off to the left presumably towards an abnormal 
termination. If the pointer is non-NULL then in the branch to the right, at loc_401591 we see 
the variable var_lC is initialized to the value of the routines parameter and var_4 is initialized 
to 0 (Figure 6.8). 

Figure 6.8 var_4 Is Initialized to 0 


B N ui 


locj* 01591: 

mou eax, [ebp+argO] 

mou [ebp+uar_1C] , eax 

mou_[ebp+uar_4], 8 


H N ui 


loc_40159E: 

cmp [ebp+uar_1C], 0 

\iz _loc_401668 




hi n ui 

mou 

ecx, [ebp+uar 1C1 

sub 

ecx, 1 

u 

loc_401668 




1 _ 


IB N Ui 1 

mou 

edx, 

, [ebp+uar 1C1 

mouzx 

eax, 

, byte ptr [edxl 

cmp 

eax, 

, 5Ah 

jnz 

loc_ 

40165A 




B N ui 

mou 

ecx, 

Tebp+uar 101 

mouzx 

edx, 

byte ptr fecx-11 

cmp 

edx. 

4Dh 

jnz 

loc_ 

40165A 




HI N Ul 1 

mou 

eax, Tebp+uar iC1 

mou 

f ebp+uar 201, eax 

mou 

ecx, Tebp+uar 201 

add 

ecx, 3Bh 

mou 

Tebp+uar 201, ecx 

mou 

edx, |ebp+uar 2U| 

mou 

[ebp+uar 24], edx 

mou 

eax, Tebp+uar 241 

mou 

ecx, [eax] 

mou 

edx, Tebp+uar 1C1 

l pa 

pax, [prix+pr.x*?] 

cmp 

eax, febp+arq 01 

jnb 

short loc_40165A 


MR u l 
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Continuing down through the function, we arrive at loc_40159E, which appears to also 
start the mark of a loop. This loop is made more obvious if you’re looking at the function in 
its entirety in the graphs; just remember this as being the loop back point, as you can tell by 
the arrow coming from the upper right back to Ioc_40159E. As far as functionality, what we 
see in this box is that the pointer copy of the routine parameter is tested against NULL again 
and then if it is, a branch occurs to the left, otherwise the routine branches to the right. From 
there we move down to the third box where the var_ 1C pointer is decremented by one and 
tested against zero, again branching to the left if the condition is true. From there, in the fourth 
box down we see the pointer dereferenced and checked against the byte 0x5A; if a match is 
found, then it moves to the fifth box, if not it branches to the left. 

If there was a match, the byte prior to this is checked against the value 0x4D, meaning 
that were looking for the 16-bit sequence 0x4D5A. As you may recall, this corresponds to 
the ASCII letters MZ, which is how a DOS header starts. Without looking further, we can 
make an educated guess that the function appears to be walking backwards through memory 
attempting to find the beginning of the file—this also explains the use of SHE as in theory 
it would be possible to touch bad memory and crash. We’ll continue looking through the 
routine though; it would be bad practice to just presume based on so little. 

Next, in the final visible box, if a match against 0x4D5A was found, the pointer is copied 
into var_20, and then incremented by 0x3B and this new pointer is copied into var_24. Finally, 
what we see next is the var_lC, or the pointer to the MZ summed with the value from i >ar_24 
( var_ 1C + 0x3B) plus two. This incidentally corresponds to the offset of the PE header in the 
DOS header and it would appear that the final box attempts to find the start of the PE header 
and then compares that pointer to arg_0 and branches dependent on result, presumably towards 
another abnormal exit (see Figure 6.9). 
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Figure 6.9 Match Against 0x4D5A 


HNlU 1 

mou 

ecx, [ebp*uar_1C] 

sub 

ecx, 1 

mou 

[ebp+uar_20], ecx 

niou 

edx, [ebp+uar_2b] 

mou 

eax, [ebp*uar_20] 

add 

eax, [edx] 

mou 

[ebp+uar_20], eax 

mou 

ecx, [ebp*uar_20] 

mouzx 

edx, byte ptr [ecx] 

cmp 

edx, 50h 

jnz 

short loc_40165ft 


p n ui i 

mou 

eax, [ebp+uar_20] 

mouzx 

ecx, byte ptr [eax*1] 

cmp 

ecx, 45h 

jnz 

short loc_*i0165ft 




H N Ul l 

mou 

edx, 

[ebp*uar_20] 

mouzx 

eax, 

byte ptr [edx+2] 

test 

eax, 

eax 

jnz 

short 

loc_40165ft 


T 




p n ui i 

mou 

ecx, [ebp+uar 20] 

mouzx 

edx, byte ptr [ecx*3] 

mou 

eax, [ebp+uar_20] 

mouzx 

ecx, byte ptr [eax+2] 

cmp 

edx, ecx 

jnz 

short loc_40165ft 


h n m _ 

nou edx, [ebp+uar_20] 

add edx, 4 

nou [ebp+uar_28], edx 

nou eax, [ebp+uar_28] 

nouzx ecx, word ptr [eax] 
cnp ecx, 14Ch 


jnz short loc_40165ft 



I 

z 


J1 

" 

r 

_ Ur —^ 

IP N LU 1 



loc_40165fi: 

nou edx, [ebp+uar_1C] 

sub edx, 1 

nou [ebp+uar_1C], edx 

jnp loc_40159E 

- - | "- 
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Moving into the next section, we see the same theme continued; in the first box we see a 
comparison against 0x50 followed by a check for 0x45.This in turn means it’s looking for 0x4550. 
If there is a match, the next two boxes check to see if the sequence is followed by two zero bytes, 
meaning a full match thus far requires the sequence 0x45500000, or PE\0\0, which is of course 
the magic value for a PE header. Finally, as the last check in this section we see that they test 
against the value 0xl4C, which corresponds to IMAGE_FILE_MACHINE_I386. All of this 
and previous sections make sense and it appears that the argument to the routine is a pointer 
that is an offset into an executable image. Given that input, the routine walks backwards through 
memory attempting to identify the start of the DOS header. Once found, it uses this information 
to locate the PE header and performs other light verifications of the data. 

As we can see in the second to last section of code, if a total match is found, we branch 
off to the left, and if not then we jump to the location loc_40165A, which takes the pointer, 
decrements it by one and repeats the loop. Now that we know the body of the routine, let’s 
examine the branches we haven’t yet looked at and also take a look at the return value. 


Figure 6.10 Jump to loc_401668 



In Figure 6.10, starting from the top right, at loc_40165A, we see that if a match was 
indeed found then we jump to loc_401668, which modifies var_4 and then passes control 
to loc_401694 which retrieves the base address, decrements it by one and then hands 
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control to the cleanup portions of the routine. This means that the return value is a 
pointer to the base address of the image in question. We also see, on the far left, the result 
of the initial test of arg_0, which if it was NULL is the value returned. Thus we can conclude 
that, given a pointer, this routine finds its base address, returning that pointer or NULL 
on error. 

Now, after reviewing this routine, let’s take a look at the main() routine again and see what 
becomes apparent. We’ve renamed this routine to FindBaseAddr() as that is the functionality 
it provides (Figure 6.11). 


Figure 6.11 FindBaseAddrO 

push offset _main 

call findBaseflddr 

add esp, 4 

mou ModuleBasePtr, eax 

cmp ModuleBasePtr, 0 

jnz short loc_4° 222 F 



loc_40222F: 

mou edx, ds:Sleep 

push edx 

call FindBaseflddr 

add esp, 4 

mou Ker^iel32BasePtr, eax 

cmp Kernel32BasePtr, 0 

jnz short loc_402256 

--r~- 


Updating this information in IDA, we can plainly see that these two invocations find the 
base address of the current module and the Kernel32 module, respectively. We’ve updated the 
variable names to reflect this, calling the pointers the ModuleBasePtr and Kernel32BasePtr for 
each. We now can make a guess at what is going on, as it’s not typical for code to manually 
find its own base through this method, and totally unnecessary to find Kernel32.We can probably 
safely bet this is an attempt at obfuscating library calls. Following the routine’s return, we see 
both pointers tested against NULL, branching to the left if this is true. Let’s not consider 
these error branches yet, and examine the scenario where both calls to FindBaseAddr() succeed. 
(See Figure 6.12). 
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Figure 6.12 loc_402256 


HNul 

loc 1)02256: 


pusha 



nou 

ebp. 

Kernel32BasePtr 

nou 

eax, 

[ebp+argL34] 

nou 

edx. 

rebp*eax«arq 701 

add 

edx, 

ebp 

nou 

ecx, 

[edx+18h] 

nou 

ebx, 

[edx+20h] 

add 

ebx, 

ebp 



H N \jl 


lOC_40226E: 

jecxz short loc_402207 

I ~T~ 


IH N Ld 

dec 

ecx 

nou 

esi, [ebx*ecx*i>] 

add 

esi, ebp 

xor 

edi, edi 

xor 

eax, eax 

cld 



H N Hi 


lOC_40227B: 

lodsb 

test al, al 

jz_short loc_482287 


IH N Ld 1 

nou 

ebx, [edx+24h] 

add 

ebx, ebp 

nou 

cx, [ebx+ecx*2] 

nou 

ebx, [edx-*1Ch] 

add 

ebx, ebp 

nou 

eax, [ebx+ecx*4] 

add 

eax, ebp 

nou 

dword_404428, eax 


1 | | T - 

rot* 

edi, ODh 

add 

edi, eax 

jmp 

short loc_40227B 


hnlu _ 

loc_402287: 

cmp edi, 0EC0E4E8Eh 

jnz short loc_40226E 


Moving on through the code, at loc_402256, which is where control is handed after the 
check of the return value from the second call to FindBaseAddr() if the base address for 
kernel32.dll was found successfully, we first see that a pusha instruction is executed (Figure 6.13). 
Some may argue with me on this, but there are instructions like pusha that I don’t see the 
compiler generate often and so I often suspect when looking at that code that it may have been 
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hand-written. I know, because I wrote the source in this case, that this is true, but it’s an 
observation that I think is generally true; your mileage may vary of course. 

At any rate, we see the Kernel32 base pointer retrieved and it has arg_34 and arg_70 added 
to the base; we then see offsets 0x18 and 0x20 retrieved from that offset and 0x20 is added to 
the base pointer. From this point, we could make a guess at what the function does based on 
what we’ve already seen and the parameters and offsets being worked with. Plus, if you’ve done 
any Windows exploitation, the entire code sequence should look familiar to you, but we’ll 
take a look a little further because something interesting occurred here. 

If you look at the first box, loc_402256, you’ll notice that arg_34 and arg_70 are used. 
There are two things that make this odd: the first is that we’re inside the main() routine, and 
there was no arg_34 or arg_70. If there were, these should be expressed as offsets from argv 
or envp .The second is that these values are only read from, so by looking at this statically we’re 
not even positive what these offsets will be exactly. This is actually a pretty good thing for 
the person employing anti-reversing; in order for me to really know what’s going on in that 
section of code, I’m going to need to look at it in a debugger and so, unless I skip this part, 
static analysis stops. In my opinion, what makes this good is that now they can actually take 
an active role in attempting to complicate the reverser’s life, as opposed to a passive/static 
role that can be achieved when being viewed under a disassembler. (See Figure 6.13). 


Figure 6.13 Viewing under a Disassembler 


.text:00402256 

pusha 




.text:00402257 

nou 


Kernel32BasePtr 

.text:00402260 

nou 

edx, 

febp+eax+arq 701 

.text:00402264 

add 

edx. 

ebp febp*arq 341=000000F0 


.text:00402266 

nou 

ecx, 

[edx+IBIdb 

0F 0h ; = 


.text-.00402269 

nou 

ebx, 

[edx+20rdb 

0 


.text:0040226C 

add 

ebx. 

ebp db 

0 


.text:0040226E 



db 

0 


.text:0040226E 

loc 40226E: 

db 

OEh 

CODE XREF: main<-3B0J] 

.text:0040226E 

jecxz 

short loc 402db 

IFh 


.text:00402270 

dec 

ecx 

db 

OBflh ; } 


.text:00402271 

nou 

esi, 

[ebx+ec>db 

OEh 


.text:00402274 

add 

esi, 

ebp db 

0 


.text:00402276 

xor 

edi, 

edi db 

0B4h ; | 


.text:00402278 

xor 

eax, 

eax 



.text:0040227ft 
.text:0040227B 
.text:0040227B 
.text:0040227B 
.text:0040227C 
.text:0040227E 
.text:00402280 
.text:00402283 
.text:00402285 
.text:00402287 
.text:00402287 
.text:00402287 
. text:00402287 
.text:0040228D 
.text:0040228F 
.text:00402292 
.text:00402294 
.text:00402298 
.text:0040229B 
.text:0040229D 


cld 

loc_40227B 
Lodsb 
test 

jz 

ror 
add 
jnp 


al, al 

short loc_402287 
edi, 00h 
edi, eax 

short loc 40227B 


loc 402287: 


cnp 
jnz 
mou 
add 
nou 
no v 
add 
nou 


edi, 0EC0E4E8Eh 
short loc_40226E 
ebx, [edx*24h] 
ebx, ebp 
cx, [ebx+ecx*2] 
ebx, [edx+lCh] 
ebx, ebp 

eax. Tebx+ecx*41 


4P General registers ' _ 1 — 11 I 

E/iX 773G0000 l4|kernel32 .dll :saued_Fp 

CF 0 1 

EBX 00000000 1 

PF 1 

1 ECX 3B25B610 f 

AF 8 

j EDX;773C00F4 l4|kernel32.dll:773CO0FJ| 

ZF 0 

ESI 00403302 l.j.rdata:00403302 

SF 0 j 

1 EDI O012FDA6: L*|stack[ 00000D4C] : 0012FDB6 

TF 0 

EBP 773C000O 4|kei-nel32 .dll :saued_Fp 

IF 1 

1 ESP: 0812FBE«4|s tack[00000D4C] : D012FBE8 

DF 0 

1 EIP 0040225D la|_main*-3BD 

OF 0 

I EFL 000002 06 
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When we look at it in the debugger, we find that this view is not all that much more 
helpful. However, if we double click on arg_34 and view that memory, what’s going on becomes 
a little more clear (Figure 6.14). 


Figure 6.14 arg_34 



kernel32.dll:77300000 
kernel32.dll:773C0000 
kernel32.dll:773C0000 
kernel32.dll:773C00O1 
kernel32.dll:77300002 
kernel32.dll:773C0003 
kernel32.dll:773C00O4 
kernel32.dll:77300005 
kernel32.dll:77300006 
kernel32.dll:773C0007 
kernel32.dll:77300008 
kernel32.dll:773C0009 
kernel32.dll:773C000A 
kernel32.dll:773000OB 
kernel32.dll:773C00OC 
kernel32-dll:773C0000 
kernel32.dll:773C000E 
kernel32.dll:773C000F 
kernel32.dll:773C001O 
kernel32.dll:773C0011 
kernel32.dll:77300012 
kernel32.dll:773C0013 
kernel32.dll:773C0014 
kernel32.dll:77300015 
kernel32.dll:77300016 
kernel32.dll:77300017 
kernel32.dll:77300018 
kernel32.dll:77300019 
kernel32.dll:7730001A 
kernel32.dll:773C001B 
kernel32.dll:773C001C 
kernel32.dll:773C001D 
kernel32.dll:773C001E 
kernel32.dll;773C0O1F 
kernel32.dll:7730002O 
kernel32.dll:77300021 
kernel32.dll:773C0O22 
kernel32.dll:773G0023 
kernel32.dll:773C0024 
kernel32.dll:773C0025 
kernel32.dll:773C0O26 
kernel32.dll:773C0027 
kernel32.dll:77300028 
kernel32-dll:773C0029 
kernel32.dll:773C002A 
kernel32.dll:773C0028 
kernel32.dll:773CO02C 
kernel32.dll:7730002D 
kernel32.dll:773C002E 
kernel32.dll:773C002F 
kernel32.dll:773C0030 
kernel32.dll:77300031 
kernel32.dll:77300032 
kernel32.dll:773C0033 
kernel32.dll:7730003^ 
kernel32.dll:773C0035 
kernel32.dll:773C0036 
kernel32.dll:773C0037 
kernel32.dll:77300038 
kernel32.dll.77300039 
kernel32.dll:7730003A 
kernel32.dll:773C003B 
kernel32 .dll :773[C003C 


assume cs:ke 
;org 7730001 

sauedfp db 
db 5Ah , 2 
db 90b E 
db 0 

retaddr db 
db 0 

db 0 

db 0 

db 4 

db 0 

db 0 

db 0 

db 0FFh 
db 0FFh 
db 0 

db 0 

db 0B8h ; * 
db 0 

db 0 

db 0 

db 0 

db 0 

db M 

db 0 

db 40h : («• 

db 0 

db 0 

db O 

db O 

db 0 

db 0 

db 0 

db d 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db 0 

db OF 0b ; = 
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When we view the pointer that IDA is calling arg_34, it becomes clear that it’s an offset 
from the base ofKernel32.dll, specifically the offset 0x3C, which as you may recall from 
FindBaseAddr() is the offset from the base to where the offset to the PE header is specified. 
This makes sense, a lot more sense than arg_34 implies. Now you may think that this is a 
great way to obfuscate intent, and indeed it works with limited success and serves mostly as 
an annoyance. However, it should be noted that other debuggers—such as OllyDBG—may 
not have the same issues. For instance, Figure 6.15 shows a screenshot from the ImmunitySec 
debugger, which is a rebranded OllyDBG with Python glued to it. 


Figure 6.15 ImmunitySec Debugger 


00402251 

00402256 

0O4O225? 

0040225D 

00402260 

004O2264 

00402266 

00402269 

O040226C 

0040226E 



WORD PTR OS:C4044243 

WORD PTR SS:CEBP+3C3 
WORD PTR SS:CEBP+EOX+78] 

BP 

WORD PTR DS:CEDX+183 
WORD PTR OS:CEOX+203 
BP 

ORT ersvr.00402207 


As you can see here, the code is represented correctly in this debugger and its purpose is 
pretty clear. The main reason this occurs in IDA is because the code fiddles with the EBP 
register (generally speaking, another tell-tale sign of inline assembly). This in turn causes IDA 
to confuse the offset and think it’s a function parameter. Another reason that could have a 
noticeable impact, although it is quite likely that fiddling with EBP was enough, is that this 
was actually a function that was inlined. Either way, after a brief step into the debugger, we 
know that these two mop’s are actually taking offsets 0x3C and 0x78 from the Kernel32 base, 
respectively. (See Figure 6.16). 


Many people find using IDA's debugger awkward at best and prefer to use 
one of the many other debuggers available. There are several others. For 
instance, WinDBG is put out by Microsoft and is a ring-0 debugger, and there 
is the slowly dying Softlce which was another ring-0 debugger, and a longtime 
cracker favorite. Flowever, lack of support for the application and changes in 
the ways that Windows operates have slowly caused Softlce to die off due to 
operability issues. 

Another debugger is OllyDBG. This has long been a staple of the reversing 
communities, largely because it's simple to use, has a fairly intuitive interface 
and is free. Semirecently, a security firm named Immunity Sec. purchased some 
form of rights to OllyDBG and combined it with the ability to script Python 
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plug-in's, along with other tweaks, and rereleased it. Immunity Debugger is 
fairly useful if you're doing exploit development, not only because of scripting 
capabilities, but also because it ships with useful scripts and features such as 
identification of heap metadata and so on. If you do much exploit development 
on Windows platforms and haven't at least tried Immunity's debugger offerings, 
you really should. 


Figure 6.16 Offsets 0x3C and 0x78 from the Kernel32 Base 
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At any rate, fixing the misrepresentation by IDA is easy enough; by right-clicking on the 
variable name we are prompted with a list of different representations, the first option being 
the correct one. Thus, we really didn’t need to use the debugger. Considering the rest of the 
code here, this is actually a fairly familiar sequence of code that, as far as the author is aware 
of, employs a technique first publicly divulged by a Polish hacker group named the Last Stage 
of Delirium (LSD) and then later expanded upon and reiterated upon in a paper by Skape 
and the nologin crew, “Understanding Win32 Shellcode.’’What we see is that the PE header 
is found at offset 0x3C, then this offset plus the base plus 0x78 yields the Exports data directory. 
The rest of the code is simply iterating over Exports and taking the names and hashing them 
with a ror instruction. This result is then compared with another 4-byte hash and, if it 
matches the export in question, has been found. 

In other words, this is just a position independent way of finding a DLLs export without 
depending on any other APIs. This is often used during the “bootstrapping” process of shellcode. 
It’s typically used to find the address of functions like LoadLibrary() and GetProcAddressQ. 
Consequently, if you look at loc_402287 you will see a cmp of EDI with the constant value 
0x0EC0E4E8E.This is the hash that this code is looking for and if you simply pop it into 
Google (or run it through a debugger) you would find that this is the hash that corresponds 
to LoadLibraryAQ. The code in the second to last box from the bottom is where a match was 
found, and thus we can rename the variable dword_404428 to be a pointer to LoadLibraryAQ and 
move on. I didn’t really dig into the details of the algorithm here, mostly due to space 
requirements, but if you’re interested I strongly advise that you look up either the paper by 
Skape or LSD. 
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Figure 6.17 Kernel32 


loc_4022A7: 


popa 



pusha 



mou 

ebp. 

Kernel32BasePtr 

mou 

eax. 

[ebp+3Ch] 

mou 

edx, 

[ebp+eax+78h] 

add 

edx. 

ebp 

mou 

ecx, 

[edx+18h] 

mou 

ebx. 

[edx+20h] 

add 

ebx. 

ebp 


a n i .a 


3 


1OC_4022C0: 

jecxz short loc_4022F9[ 



a Nui l 

dec 

ecx 


mou 

esi. 

[ebx+ecx»4] 

add 

esi. 

ebp 

xor 

edi, 

edi 

xor 

eax, 

eax 

cld 




Nul 


J 


l0C_4 022CD: 
lodsb 

test al, al 

jz short loc 4022D9 


T 


BN Id 1 

ror 

edi, 


add 

cun 

eax 

jnp 

short 



HN id 


loc_4022D9: 

cmp edi, ubf ChhIi 

jnz short loc_4022C0| 


L 


INul 


mou 

ebx. 

[edx+24h] 

add 

ebx. 

ebp 

mou 

cx. 

[ebx+ecx*2] 

mou 

ebx, 

[edx+ICh] 

add 

ebx. 

ebp 

mou 

eax, 

[ebx+ecx*4] 

add 

eax , 

ebp 

mou 

dword_40441C, eax| 
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Moving along, we notice a startling familiarity in the next section of code (Figure 6.17), 
almost to the point that you may wonder if I reposted the wrong image! I assure you I did 
not. What we see here is another walk through Kernel32 in the same manner in an attempt 
to find another exported function. This time the hash in question can be found at loc_4022D9 
and has a value of 0x7C0DFCAA. Once again, either via Google or a debugger, you would 
find pretty quickly that this is the hash value for the GetProcAddressQ function; thus this 
section of code just locates that pointer, and saves it at dword_40441C, which we will rename 
to GetProcAddressPtr. One other thing the reader might note is that this section of code did 
not exhibit the same oddity as earlier when extracting offsets 0x3C and 0x78. This is simply 
the result of my changing its representation already. 


Figure 6.18 loc_4022F9 


HNul 

Xoc 4022F9: 

popa 


push 

Oflh 

lea 

eax, [ebp+oar_6C] 

push 

eax 

call 

sub 901400 

add 

esp, 8 

lea 

ecx, rebp<-uar 6C1 

push 

ecx 

call 

LoadLibraryfiPtr 

nou 

dword 40442C, eax 

push 

onh 

lea 

edx, rebp«-uar 6C1 

push 

edx 

call 

sub 4014D0 

add 

esp, 8 

crop 

dword 40442C, 0 


short loc 402345 


Immediately following the previous code, in Figure 6.18, we find loc_4022F9 in which 
we can see a call to another new subroutine whose return value is stored in i >ar_6C. Following 
that we see a call to LoadLibraryAQ so it’s a pretty good guess that this likely decrypts or decodes 
some of that stack data we looked at first. Furthermore, we find that we’ll save the return 
value from LoadLibraryAQ in dword_40442C. Finally, we get another clue as to what to expect 
for the encryption/encoding, as after the usage of the string we see that a call to sub_4014D0 
is made again with the same argument of var_6C , at least implying that we’re probably going 
to be looking at a symmetric cipher of some sort. We'll take a brief look at this implementation 
just to get an idea of what’s going on in there, but once you’ve confirmed that it’s just some 
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sort of string obfuscation scheme that encrypts/decrypts itself, it’s typically fastest to simply 
let it do its thing and copy the results out. 

That said, you should probably at least read the source of the function you’re going to 
step over to ensure it does what you think it does; we’ll largely leave this as an exercise for 
the reader however, as the book gains little from a detailed analysis of the crypto employed 
and it would take up a significant amount of space in a chapter that’s already tight. Plus, it’s 
trivial but not trite, so it will work out as a good educational exercise for the inspired reader. 


Figure 6.19 Reloading the Debugger 

* .text:O04022FC lea 

* .text:004022FF push 

* .text:00402300 call 

* .text:00402305 add 

__’ .text:00402308 lea 

fau » gjjgg 

* .text:O040230C call 

* .text:00402312 nog 

* .text:00402317 push 

* .text:00402319 lea 

* .text:O040231C push 

* .text:0040231D call 

* .text:00402322 add 

* .text:00402325 cnp 

* .textI0040232C jnz 

* .text:0040232E push 

* .text:00402330 lea 

* .text:00402333 push 

* .text:00402334 call 

* .text:00402339 add 

* .text:0040233C push 

* .text:0040233D call 

* t-nvi--nniin9'lli9 aria 


Reloading the debugger, letting the program do the hard work and decrypt the string 
for us, what we see post calling sub_4014D0 is the hex sequence seen in Figure 6.19. If you 
look it up, this corresponds to the ASCII mapping for the NULL terminated string “rpcrt4. 
dll”—the library required to make RPC calls in Windows. This makes sense, of course, given 
that we know this to be an RPC server from our earlier analysis. This also largely pieces the 
puzzle together, as we can likely guess that the purpose of the GetProcAddressf) pointer. Once 
again, to conserve space and leave something to do for the reader, we’ll leave the rest as an 
exercise. 


eax, [ebp+oar_6C] 
eax 

sub_4014D0 
esp, 8 

ecx, [ebp*uar 6C1 




LoadLibraryftPtr 
dword_40 | * 1 * 2C , eax 
Oft h 

edx, [ebp+gar_6C] 
edx 

sub_4014D0 [ebp*uar_6C]*72637072 

esp, 8 gar_6C dd 72637072h 

duord_48442C, gar_68 dd 642E3474h 
short loc_4023gar_64 dw 6C6Ch 

17h gar 62 db 0 _ 

eax, [ebp*gar_24] - 
eax 

sub_4014D0 
esp, 8 

eax ; char * 

sub_401Efi0 

QCn Jl 
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Summary 

To review the techniques described in this chapter, we’ve seen the code base make-up change 
drastically and the complexity of the program increase rapidly as well. We’ve seen how easy it 
is to remove the string constants and similar. The interested reader who finds this particular 
section interesting might enjoy reading more about things like overwriting pointers in the Import 
Address Table (IAT), or copying system functions into user allocated space to avoid breakpoints 
on functions. Another technique that originated from the virus-writing world is a technique 
called entry point obscuring or EPO. Traditional viruses would modify the entry point in the 
executable header and typically append themselves to the executable file. This of course yielded 
a tell-tale sign of infection and gave anti-virus an easy target. As a result, EPO viruses started 
to appear. EPO viruses, instead of modifying an entry point, will scan the executable section 
for a jmp or similar and modify that to hand control to the virus elsewhere. This same technique 
could be used to take advantage of the fact that IA-32 machines have limited debugging 
support by entering into system libraries 5 to 10 bytes into the function instead of at its original 
entry point. 
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Solutions in this chapter: 

■ Tracing Execution Flow from a Read Event 

■ Determining the Structure of a Protocol 

■ Determining if the Protocol has any 
Undocumented Messages 

■ Use IDA to Determine the Functions that 
Process a Particular Message 
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The Protocol Problem 

ItB not uncommon to be presented with an executable where the protocol is either partially 
unknown or completely unknown. As a reverse engineer, itis your job to either Ygure out 
the protocol for compatibility or to check a program for any hidden features that may cause 
security problems. In this chapter weill cover tracking a protocol through a binary and 
recovering its message structure. 


Protocol Structure 

Most protocols are streams of discrete messages meant to be interpreted individually. There 
are exceptions to this rule. HTTP, for instance, dumps off a mostly unstructured request and 
then gives an unstructured reply. FTP uses a text-based control channel and establishes a 
separate TCP session for each Yle transferred. These are in the minority of protocols youill 
have to reverse. 

If a reverse engineer doesnit have access to either of the executables, the protocol can 
be reversed from only the raw bytes on the wire. If a reverse engineer only has access to 
the binary without the ability to run it, the protocol can eventually be extracted from the 
executables. In most cases, the engineer will have access to both the binaries and a work¬ 
ing implementation. A hybrid approach is the fastest way to solve the problem. The bytes 
on the wire give the reverse engineer a quick picture of how the protocol is structured, 
but, in the end, any features not exercised by the client or server will have to be extracted 
from the binary. If the reverse engineer has the time, the binary always gives the most 
accurate view. 

Framing and Reassembly 

Every protocol needs to know where one message ends and the following message begins. 
This is commonly referred to as framing. Most protocols can survive getting out of synchro¬ 
nization with the other end of the connection. If the sender thought a message was 30 bytes 
long and the receiver interpreted the message as 20 bytes long, the receiver might try to 
interpret the remaining 10 bytes as a new message, resulting in a corrupt message and the 
connection being dropped. 

TCP/IP doesnit guarantee a message will remain together as it makes its way across the 
Internet. The message may be broken up into smaller pieces along the way. The mechanics 
are unimportant to the programmer. What is important is that a single call to a read() 
function may return a whole message or some piece of the message. It might also include 
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more than one message. Ms up to the application to make sure it has read a full message into 
the buffer before continuing. lid like to be able to say that most programs do this well. 
Unfortunately, most programs youill encounter in the wild do a very poor job of 
reassembling and then parsing the messages. 

Most small or quickly written programs will assume that messages are transmitted whole 
across the Internet.Their basic block diagram looks something like Figure 7.1. 


Figure 7.1 Basic Block Diagram of a Small Program 



If a message is broken up en route, the program will either crash while parsing the mes¬ 
sage or reject the message as incomplete, depending on how the function was implemented. 
Programs written this way still work surprisingly well. When a message is fragmented, the 
message is returned to the program broken across two reads from the socket. Both the 
messages are discarded as invalid, but the next message (if it wasnit also broken up) will be 
received and processed correctly by the system. 

A popular instant messenger client is among the programs that implement this kind of 
loop. Spotting these programs is easy. Simply bring up a proxy between the client and the 
server and break the messages up in small pieces. If the messages are ignored, you have a 
program that assumes the messages will be delivered as a unit. 

Larger commercial programs often buffer reads into a read buffer and then shift the read 
buffer down as full messages are received. Their basic block diagram looks like Figure 7.2. 

This is a perfectly correct loop and easy to trace through the system. The receive buffer tends 
to be allocated as a global or on the heap, and if ids properly implemented works in all cases. 


www.syngress.com 



























140 Chapter 7 • Walkthrough Four 


Figure 7.2 Basic Block Diagram of a Large Commercial Program 



New Message 

New Message 



Receive Buffer 



Receive new data 





Is Message 
Complete? 





ProcessMessage 





Shift Buffer Down 




Receive Buffer 


Self Similarity 

Protocols are self-similar, meaning that since operations are repeated over the life of the 
session, the bytes transferred over the wire also repeat. Nearly every protocol has a common 
protocol header. That header will appear in every message on the wire. At a minimum, the 
header will contain a length and a message type. The parser needs to know how big the 
current message is and how to parse it. The following list shows some of the Yelds often 


found in 

the base protocol header. 

■ 

Magic Number 

■ 

Sequence Number 

■ 

Timestamp 

■ 

Data or Section Lengths 

■ 

Session ID 

■ 

Number of Submessages 

■ 

Error Code 

■ 

Random Nonce 
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When starting to make sense of a protocol, it helps to have examples of the various 
packets in front of you for reference. I always print out the hex dumps so I can annotate 
them as I go. Below are the Yrst few packets from the example used in this section. live 
highlighted the packets going from the client to server in bold and the packets going from 
the server to the client in italic. 


DE AD BE EF 00 18 01 00 76 B7 0B 5A 42 DD 54 B9 6B E8 IE 47 44 D9 67 C3 
DE AD BE EF 00 18 02 00 Cl 06 45 18 90 51 5D 71 44 46 D7 21 B6 4C 01 73 
DE AD BE EF 00 18 01 00 45 EB 2E 08 42 68 22 72 60 E9 IB 32 16 FA 45 EB 
DE AD BE EF 00 18 02 00 2E 08 42 68 22 72 60 E9 IB 32 16 FA 40 AB 55 AD 
DE AD BE EF 00 18 01 00 64 D4 7F 88 7E El 5A AA 21 46 49 3D E3 22 7E IE 
DE AD BE EF 00 18 02 00 79 18 D2 6C D7 3D C9 61 60 7B 02 00 DC 4F 40 59 

DE AD BE EF 02 08 03 01 AF 59 3E 31 ED 45 FD 02 E3 26 A1 IB 08 12 19 05 16 82 
59 26 3B 90 77 

DE AD BE EF 02 08 03 00 77 0D 33 A4 03 19 4D FI 62 F5 IF B2 20 CB 37 82 25 87 
46 0E 6E 8A 56 

DE AD BE EF 02 08 04 00 77 0D 33 A4 03 19 4D FI 62 F5 IF B2 20 CB 37 82 25 87 
46 0E 6E 8A 56 

DE AD BE EF 02 08 04 00 AF 59 3E 31 ED 45 FD 02 E3 26 A1 IB 08 12 19 05 16 82 
59 26 3B 90 77 

DE AD BE EF 00 08 01 02 00 00 00 00 


At this point, we have no idea how many different types of messages are in this dump and 
we have no idea how long the base header is. Looking at it column by column, we can start 
making some guesses. The Yrst four bytes have the same value in every packet. Since they spell 
DEADBEEF, we can safely assume that theyire a magic number at the beginning of the protocol. 
The shortest packet in this dump is 12 bytes long. Itis safe to say that the base header is smaller 
than the smallest message observed. That leaves us with the start of a structure that looks like this: 

struct base_header{ 


int 1 

MagicNumber; 

/*Always OxDEADBEEF*/ 

char 

unknown1; 

/*00,02*/ 


char 

unknown2; 

CO 

o 

-X 

18*/ 


char 

unknown3; 

/*01, 

02, 

03, 04*/ 

char 

unknown4; 

/*00,01,02 

*/ 

char 

unknown5; 

/*Lots 

of 

possibilities*/ 

char 

unknown6; 

/*Lots 

of 

possibilities*/ 

char 

unknown7; 

/*Lots 

of 

possibilities*/ 

char 

unknown8; 

/*Lots 

of 

possibilities*/ 
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The packets have to be read off the network and into the program. There are a limited 
number of API calls that accomplish the task. The program can directly make the system calls 
itself, but thatis only encountered in malware. The place(s) where the packet is read off the 
wire is always a good place to start your analysis. The following is a list of common API calls 
capable of reading trafYc off a network socket. 

■ read/write 

■ recv/send 

■ recvfrom,/sendto 

■ WSARecv/WSASend 

■ WSARecvFrom/WSASendTo 

■ ioctl 

■ ioctlsocket 

■ WSARecvDisconnect/WSASendDisconnect 

■ WSARecvEx/WSASendEx 

■ recvmsg/sendmsg 

■ W S ARecvMsg/W SASendMsg 


Figure 7.3 Import Table of Executable Showing Reference to WSARecv 


la Imports [ ■=■ ' S II <£3*l 


Address 

0 rdinal 

Name 

Libraij ■* 

IB^ 004020 . 


onexit 

MSVC 

6^00402004 

116 

WSACleanup 

WS2_ 

B*004020C8 


WSAAccept 

WS2_ 

6*004020... 

12 

inet ntoa 

WS2 

6* 004020... 

115 

WSAStarlup 

WS2 

6*004020 . 


WSASocketW 

WS2_ 

B* 004020 . 

2 

bind 

WS2 

6* 004020. . 

13 

listen 

WS2_ 

Ilaryuwuiui u 



_' - liil 

6*004020E4 111 

WSAG etLaslError 

WS2_ 

6* 00402OE 8 14 

ntohl 

WS2_ c |i 

6*004020.. 

15 

nlohs 

WS2_[j 

6*00402OFO 

3 

dosesocket 

WS2_ 

B& 004020F4 

8 

hlonl 

WS2 ’ 

1 4 


jn 

I 


Line 55 of 62 
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The import table of the executable only has a reference to WSARecv so that serves as in 
ideal starting point for the analysis of the protocol (see Figure 7.3). The WSARecv call reads the 
data into a buffer on the stack. It can read up to 0x4000 bytes in one read as shown in Figure 7.4. 

Figure 7.4 WSARecv Call 


a n m 

push 

4B00h ; sizet 

push 

0 ; int 

lea 

pcx, [ebp+uar_4018] 

push 

ecx ; uoid * 

call 

menset 

add 

esp, OCh 

mou 

[ebp+Buffers .len ], 4000h 

lea 

edx, [ebp+uar 4018] 

mou 

[ebp+Buffers.buf], edx 

push 

0 ; lpConpletionRoutine 

push 

0 ; lpOuerlapped 

lea 

eax, [ebp+Flags] 

push 

eax ; lpFlags 

lea 

ecx, [ebp+NunberQfButesRecud] 

push 

ecx ; lpNunberUfBytesHecud 

push 

1 ; dwBufferCount 

lea 

edx, [ebp+Buffers] 

push 

edx ; lpBuffers 

mou 

eax, [ebp+s] 

push 

eax ; s 

call 

ds:USARecv 

mou 

[ebp+uar 401C], eax 

cmp 

[ebp+uar_401C], 0 

jz 

short loc 40128C 


— 


Right after reading into a buffer on the stack, the program immediately checks to see if 
the number of bytes read is less than eight. If its fewer than eight bytes, it jumps to the exit 
function, as shown in Figure 7.5. 


Figure 7.5 Exit Function 


SHJj! 


loc_40128C: 

cmp [ebp^HijuiberOFBytesRecud] , 8 

jnb short loc_4Q12AA 


It then checks the Yrst four bytes to see if theyire DEADBEEF. That matches the magic 
number from the dump (the Yrst four bytes) nicely. The bytes read off the wire are passed 
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through ntohl() before theyire compared. This means the wire protocol is big-endian. It will 
also help to determine the size of Yelds.The program will need to byte-swap all two- and 
four-byte Yelds before processing them, as shown in Figure 7.6. 

If it passes the DEADBEEF test, it takes the next two bytes and checks to see if theyire 
less than eight. Packets with a small number are discarded. We now know that the two bytes 

Figure 7.6 Byte-swapping 


- 


a n hi 


loc 401200: 



lea 

ccx. 

[cbp+uar_40l8] 


CIUU 

[ebp«-udr 402U], ULX 


mu u 

Hilx, 

f ehp*v«ir 4020] 


mou 

eax, 

r edxl 


push 

eax 

; netlong 


call 

ds:n 

tohl 


emp 

eax, 

0DERDBEEFh 


jz 

short loc_h012Eh 




1 J 



* 

H N Ul 1 

luu 40I2E4: 


mou 

OCX , 

[ebp+oar 4020] 

mou 

dx. 

r ecx»4] 

push 

PflX 

; nptshort 

call 

d5:otohs 

rnouzx 

eax, 

ax 

crop 

eax, 

8 

jnb 

short loc_b01315 


following the magic number are a unit and they always have to be greater than eight. With 
the information obtained from these two basic blocks, the structure diagram can be updated. 
The number corresponds with the length of the packet, so for now Iill label the two-byte 
Yeld as the length Yeld. 


struct base_header{ 

int MagicNumber; 
unsigned short Len; 
char unknown3; 
char unknown4; 
char unknown5; 
char unknown6; 
char unknown7; 
char unknown8; 

} 


/*Always 0 xDEADBEE F * / 

/*0018,0208*/ /*Greater than 8*/ 
/*01, 02, 03, 04*/ 

/* 00 , 01 , 02 */ 

/*Lots of possibilities*/ 

/*Lots of possibilities*/ 

/*Lots of possibilities*/ 

/*Lots of possibilities*/ 
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After the length check, the processing function pulls a single byte at offset 7 into a 
register and uses the and instruction against the result. Whenever a Yeld pulled out of the 
protocol is passed to bitwise and or or operators, and the operand is a constant, itis a good bet 
that the target is a bit Yeld of some sort. In this case, if the third bit is set, the process stops 
processing the packet. Itis safe to say that byte 7 is a bit Yeld and the constant 0x04 is invalid, 
as shown in Figure 7.7. 


Figure 7.7 Use of and Operator 


i 


PI N id 


loc 401315: 



ntou 

cck , 

f cbp+uar 

402 0] 

mouzx 

edx, 

byte ptr 

fecx+?l 

and 

odx, 

4 


jz 

short 

loc_401337 


TT 


The updated struct looks something like: 


struct base_header{ 

int MagicNumber; 
unsigned short Len; 
char unknown3; 

#define FLAG_NONE 
#define FLAG_INVALID 
char Flags; 
char unknown5; 
char unknown6; 
char unknown7; 
char unknown8; 


/*Always OxDEADBEEF*/ 

/*0018,0208*/ /*Greater than 8*/ 
/*01, 02, 03, 04*/ 

0x00 

0x04 

/* 00 , 01 , 02 */ 

/*Lots of possibilities*/ 

/*Lots of possibilities*/ 

/*Lots of possibilities*/ 

/*Lots of possibilities*/ 


From the packet dump, we saw that two other Ehgs were set. In most Ehgs an empty 
bitYeld means ido nothings so as a placeholder, zero is deY ned as FLAG_NONE.That 
leaves at least 0x01 and 0x02 as valid Efegs in the protocol. 
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Next up, the binary takes byte 6 and stores it into a stack variable. It compares the stack 
variable to three possible values: 0x01, 0x03, and 0x5C. In the dump, weive seen 0x01 
through 0x04, but only 0x01 and 0x03 are packets destined for the server. It would seem 
that 0x02 and 0x04 are client—^server constants while 0x01, 0x03, and 0x5C are server —> 
client constants. All other values arenit processed, as shown in Figure 7.8. 


Figure 7.8 Other Values Are Not Processed 



Depending on the value of unknown3, the binary will call one of three functions. The 
prototype for each of the called functions is the same. The Yrst parameter is a pointer to the 
raw packet we sent on the wire. The second is the number of bytes read from the network, 
and the third is the socket handle the packet was read from, as shown in Figure 7.9. 


Figure 7.9 Prototype for Each of Three Called Functions 



Later in the tutorial, weill reverse each of the three functions. For now, weill Ynish out 
the logic of the current function. The Ynal two comparisons are both against the Fig Yeld. 
The binary uses a bitwise and to check for the 0x01 bit. If the bit is set, the thread goes to 
sleep for 1000ms, as shown in Figure 7.10. 
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Figure 7.10 If Bit Is Set, Thread Goes to Sleep for 1000 ms 




The last check before the end of the processing loop is another bitwise and checking for 
the 0x02 bit in the bitYeld. If the bit is set, the processing loop exits, as shown in Figure 7.11. 


Figure 7.11 If Bit Is Set, Processing Loop Exits 



That completes the main parsing loop of the example server. Since we didnit do 
anything with the other bytes in the struct, it can be safely assumed that they arenit part of 
the base header in the protocol and those bytes belong to some deeper part of the protocol. 
The Ynal structure appears in the code below. 

struct base_header{ 

int MagicNumber; 
unsigned short Len; 

#define PACKET_SERVER_01 
#define PACKET_CLIENT_02 
#define PACKET SERVER 03 


/*Always OxDEADBEEF*/ 

/*0018,0208*/ /*Greater than 8*/ 

0x01 

0x02 

0x03 
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#define PACRET_CLIENT_04 
#define PACKET_SERVER_5C 
char PacketType; 

#define FLAG_NONE 
#define FLAG_SLEEP 
#define FLAG_PROCESS_AND_EXIT 
#define FLAG_INVALID 
char Flags; 

} 


0x04 

0x5C 

/*01, 02, 03, 04*/ 

0x00 

0x01 

0x02 

0x04 

/* 00 , 01 , 02 */ 


For this simple protocol, all weire left with is to Ygure out what each of the three 
message types are for. The server processes messages of type 0x01, 0x03, and 0x5C. 

The 0x01 message is as good a place to start as any other, since weire trying to reverse 
the entire protocol. 

The function at 0x00401000 handles messages of type 0x01. Three parameters are passed 
to it: a pointer to the stack buffer containing the full message, the number of bytes read, and 
the socket it was read from, as shown in Figure 7.12. 


Figure 7.12 Three Parameters Are Passed to the Function 


push 

ehp 


nnu 

php, psp 


sub 

esp, 24h 


mou 

eax, duoi'd 4Bauuil 


xor 

eax, ebp 


(TOO 

[ebp+uar 4], eax 


push 

18h ; 

si 2 e_t 

push 

o ; 

int 

lea 

eax, [ebp+bul ] 


push 

eax ; 

uoid * 

call 

menset 


odd 

esp, 6Ch 


push 

ODEUDBEEFh ; 

hostlong 

call 

ds:htonl 


mou 

dword ptr [ebp+buf], eax 

push 

18h ; 

hostshort 

call 

ds:htons 


nou 

[ebp+uar 18], ax 


nou 

f ebp+uar IS ], 0 


nou 

[ebp+uar 16], 2 


nou 

[ebp+uar 241, 0 


jnp 

short 1 oc3&1B54 



The Yrst thing you should notice is that the function never touches the buffer or the 
length Yelds passed into it. It jumps straight to constructing another message in a local 
stack buffer. It Yrst zeros the buffer with memset, then byte-swaps the magic number 
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(DEADBEEF) and stores it in the buffer. Then it byte-swaps and stores a Oxl 8 in the length 
Yeld, a 0x02 in the type Yeld, and a zero in the Ehgs Yeld. Then it Ylls the next 0x10 bytes 
with random values, as shown in Figure 7.13. 

Figure 7.13 Fills Next 0x10 Bytes with Random Values 



Finally it writes it out to the socket, as shown in Figure 7.14. 

Figure 7.14 Writing It Out to Socket 



H N Ld 

loc_4 01 069: ; 

flags 

push 

D 


push 

18h ; 

len 

lea 

eax, [ebp+buf] 


push 

eax ; 

buf 

nou 

ecx, [ebp+s] 


push 

ecx ; 

s 

call 

ds :send 


nou 

febp+uar 28], eax 


cnp 

febp+uar 281, 18h 


jz 

short loc 401B96 



Basically, this generates a predetermined packet and returns it to the caller. The contents of 
the message received arenit taken into account.We now know the bytes after the header in the 
calling packet are ignored and the bytes following the 0x02 message are just random values. 
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struct 


# define 

# define 

# define 

# define 

# define 

# define 

# define 

# define 

# define 


base_header { 
int MagicNumber; 
unsigned short Len; 
PACKET_GETRANDOM 
PACKE T_RANDOM 
PACKET_SERVER_03 
PACKET_CLIENT_0 4 
PACKET_SERVER_5C 
char PacketType; 
FLAG_NONE 
FLAG_SLEEP 

FLAG_PROCESS_AND_EXIT 

FLAG_INVALID 

char Flags; 


/*Always OxDEADBEEF*/ 

/*0018,0208*/ /*Greater than 8*/ 

0x01 

0x02 

0x03 

0x04 

0x5C 

/*01, 02, 03, 04*/ 

0x00 

0x01 

0x02 

0x04 

/* 00 , 01 , 02 */ 


struct packet_get_random{ 

struct base_header BaseHeader; 
char Ignored[16]; 


struct packet_random{ 

struct base_header BaseHeader; 
char RandomValues[16]; 


The next message is processed by the function at 0x004010B0. Itis passed the same three 
parameters as the packet above: a pointer to the message buffer, the message buffer length, 
and the socket the message was read from. 

The Yrst thing it does is to make sure the message length is at least 520 bytes, as shown 
in Figure 7.15. 


Figure 7.15 Ensure Message Length at Least 520 Bytes 



mou ebp, esp 

sub esp, 214h 

mou eax, dword_403000 

xor eax, ebp 

mou [ebp*uar_8], eax 

cmp [ebp+ReadBufferLen], 520 

jnb short loc_4010D3 

= 
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The entire processing of this packet is done in a single basic block, as shown in 
Figure 7.16. 


Figure 7.16 Single Basic Block Processes Packet 


HNui 

loc 4010D3: 

uniu 

HriX, [pll|l + ReHllRllff PI'] 

nou 

[ebp+uar 2141, eax 

ini'll! 

7 nail ; si/H_l 

push 

0 ; int 

193 

ocx, [ebp+tiuf] 

push 

ecx ; uoid * 

call 

nenset 

add 

esp, 0Ch 

push 

BDEflDBEEFh ; hostlong 

call 

ds:htonl 

nou 

duord ptr [ebp+buf], eax 

push 

208h ; hostshort 

call 

ds:htons 

nou 

[ebp+uat 20C], ax 

nou 

[ebp+uar_ZUMJ , U 

nou 

[ebp+uat 20fl], 4 

push 

'i mill ; size_t 

nou 

edx, [ebp+uar 214] 

add 

edx, 8 

push 

edx ; uoid * 

lea 

eax, [ebp+uar_208] 

push 

eax ; uoid * 

call 

nencpy 

add 

esp, OCh 

push 

B ; flags 

push 

2 08h ; len 

lea 

ecx, [ebp+huf] 

push 

ecx ; buf 

nou 

edx, [ebp's] 

push 

edx ; s 

call 

ds :send 

nou 

[ebp+uar 4], eax 

cnp 

[ebp+uar 4], 208h 

jz 

short loc_40l177 


The basic block zeros a stack buffer with memset. Then it byte-swaps the magic number 
(DEADBEEF) and stores it in the Yrst four bytes of the buffer. It sets the byte-swapped 
length to 0x208, the type to 4 and the Ehgs to zero. Then it copies 512 bytes from the 
incoming packet and puts it in the outgoing packet. 
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struct base_header{ 

int MagicNumber; 
unsigned short Len; 

#define PACKET_GETRANDOM 
#define PACKET_RANDOM 
#define PACKET_ECHO 
#define PACKET_ECHOREPLY 
#define PACKET_SERVER_5C 
char PacketType; 

#define FLAG_NONE 
#define FLAG_SLEEP 
#define FLAG_PROCESS_AND_EXIT 
#define FLAG_INVALID 
char Flags; 

} 


/*Always OxDEADBEEF*/ 

/*0018,0208*/ /*Greater than 8*/ 

0x01 

0x02 

0x03 

0x04 

0x5C 

/*01, 02, 03, 04, 0x5C*/ 

0x00 

0x01 

0x02 

0x04 

/* 00 , 01 , 02 */ 


/‘must be 520 bytes long or longer*/ 
struct packet_get_echo{ 

struct base_header BaseHeader; 
char RandomData[512 ] ; 


} 


struct packet_echoreply{ 

struct base_header BaseHeader; 
char EchoData[512] ; 


} 


The last message (type 0x5C) is handled by the function at 0x00401190. Iris passed the 
same three parameters as the other message handlers: a pointer to the message, the message 
length, and the socket the message came in on. The message is handled in a single basic 
block, as shown in Figure 7.17. 

It looks like this message spawns a calculator. The Ynal protocol description is given 
below. 


struct 

base header { 





int MagicNumber; 

/*Always 

OxDEADBEEF*/ 


unsigned short Len; 

/*0018,0208*/ 

/*Greater than 

# define 

PACKET GETRANDOM 

0x01 



# define 

PACKET RANDOM 

0x02 



# define 

PACKET ECHO 

0x03 



# define 

PACKET ECHOREPLY 

0x04 



# define 

PACKET CALC 

0x5C 




char PacketType; 

/*01, 02, 

03, 

04, 0x5C*/ 
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#define FLAG_NONE 0x00 

#define FLAG_SLEEP 0x01 

#define FLAG_PROCESS_AND_EXIT 0x02 

#define FLAG_INVALID 0x04 

char Flags; /*00,01,02*/ 

} 

struct packet_get_random{ 

struct base_header BaseHeader; 
char Ignored[16]; 

} 

struct packet_random{ 

struct base_header BaseHeader; 
char RandomValues[16] ; 

} 

/‘must be 520 bytes long or longer*/ 
struct packet_get_echo{ 

struct base_header BaseHeader; 
char RandomData[512]; 

} 

struct packet_echoreply{ 

struct base_header BaseHeader; 
char EchoData[512]; 

} 

struct packet_calc{ 

struct base_header BaseHeader; 

} 

The protocol description above should be enough to implement a client. Weire now also 
sure that this is all of the functionality of the serverothere are no hidden uses the server can 
be put to. 

Hit Marking 

The example used in the Yrst part of this section is a simple one. There are only a few 
functions and all of them directly deal with the protocol. Reverse engineering the entire 
program is feasible for the example, but itis rare that a reverse engineer will have the luxury 
of reversing an entire program. Itis easy to Ynd the functions to tear apart when a programis 
call tree looks like Figure 7.18. 

Itis much more difYcult when the call tree looks like Figure 7.19. 

The Yrst question the reverse engineer must answer is which of the functions in this 
monstrous graph are used in processing messages and which ones are ignored. The most 
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Figure 7.17 Single Basic Block That Handles Last Message 


FfflNul 

; Attributes: bp-based Frane 

; int 

cdecl sub 40119B(char *ReadBuffer ,int ReadBufferLen,SOCKET s) 

sub 401190 proc near 


uar 4= 

dword ptr -4 


ReadBuFfer= dword ptr 8 


ReadBufferLen= dword ptr 

OCh 

5“ dword ptr 1Bh 


push 

ebp 


nou 

ebp, esp 


push 

ecx 


push 

i ; 

uCndShow 

push 

offset CmdLine ; 

"c:\\windows\\systen32\\calc.exe" 

call 

dsz'JinExec 


nou 

[ebp+uar 4], eax 


nou 

eax, 2 Oh 


cnp 

eax, [ebp+uar_4] 


sbb 

eax, eax 


neg 

eax 


nou 

esp, ebp 


pop 

ebp 


retn 



sob_4B1190 endp 
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common method of picking which functions to reverse engineer is referred to as hit 
marking. 

The basic concept behind hit marking is to record the path through the decision tree 
that each message takes. A breakpoint is set at the beginning of each function or basic block. 
When a breakpoint is reached, the program notes that the function was accessed and lets the 
program continue on to the next function. After the message has been fully processed, the 
list of breakpoints that were hit is sorted and duplicates are removed. This list is referred to as 
a hit list. By analyzing just the functions on the hit list, the reverse engineer should have an 
understanding of the structures that make up that particular message. 

Figure 7.20 shows a short trace from IDA before duplicates are removed. 


Figure 7.20 Short Trace from IDA 


Hi T'ici •""rviO'A 


Thread 

Addiess 

Instruction 

; 1 OQOOOE40 

text sub_401BD5 

push 

esi 

B 00000E4Q 

text.sub_401E1D 

XQf 

eait eax 

1 1 iminHit in 

lext£ijh_4m 490 

push 

fihp 

: R OOOOOE40 

,text.sub_401SCC 

crnp 

eex, dwotd_4Q2000 

B OOOOOE40 

lent iub_4015MD 

push 

ebp 

3 0000152C 

. text.sub_401000 

push 

ebp 

1 1 0000152C 

text sub_401 BCC 

cmp 

ecx. dwmd_403000 

:B 0000152 c 

Text. sub_40i 6CC 

crnp 

ecx, dword_403000 


Unfortunately, IDA doesnit have a good mechanism for building hit lists. A number of 
third-party plug-ins and applications can be purchased to make the process more useful and 
less painful, but weire going to stick to pure IDA. The rest of this chapter will walk through 
developing a hit list. 

The Yrst thing youill need to build a hit list is a list of all the functions in the program. 

If you happen to have a way of removing functions that you know youire not interested in, 
then by all means remove them. To obtain a list of functions, open your binary in IDA and 
open the function window (shift-f3). Right click and select copy (or hit ctrl-ins). IDA will 
copy the list of functions to the clipboard. Paste the list of functions into your favorite editor. 
Youill get a list something like: 

_IID_ISAXErrorHandler .text 01001308 00000010 R . 

_IID_IXMLDOMDocument2 .text 01001318 00000019 R . . . . T . 

IID ISchemaElement .text 01001338 00000009 R . 
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_IID_ISchemaAttribute .text 01001348 00000009 R . 

_IID_ISchemaModelGroup .text 01001358 0000000D R . 

_IID_ISchemaComplexType .text 01001368 0000000D R . 

_IID_ISchemaType .text 01001378 0000000D R . 

_IID_ISchemaItem .text 01001388 00000009 R . 

_IID_ISAXAttributes .text 010013A8 00000010 R . 

Use your text edit to remove everything but the function addresses. It should look 
something like: 

01001308 

01001318 

01001338 

01001348 

01001358 

01001368 

01001378 

01001388 

010013A8 

The IDA debugger has a built-in mechanism for tracing. It can be accessed from the 
Debug | tracing window or you can right-click on any instruction and add an execution 
trace. Theoretically, you could manually run through your entire function list and manually 
add a trace to each breakpoint. Four days later, your boss will probably Yre you. 

To efYciently add tracepoints to each function, we need to write a quick IDA plug-in. 
IDA plug-ins can be written in C, ruby, or python. Programming language holy wars arenit a 
good thing, so Iim going to sidestep the issue and just write the plug-in in python because I 
wanted to (not because python is better) .You can get the python plug-in from www. 
d-dome.net/idapython/. Just follow the instructions to install it. 

What we want the plug-in to do is to automate setting all those hundreds of tracepoints. 
As a simple example, you can set a breakpoint at 0x004011E8 with just a few lines of 
python. Create a Me with the following content: 

#Set a breakpoint at 0x004011E8 
from idautils import * 
ea=ScreenEA() 
ea=0x004011E8 
add_bpt(ea, 1, 4) 

Now run the Yle by hitting alt-9 and selecting it from the dialog box. (The Yle has to 
end in .py ifyouire using the python bindings.) A breakpoint should now be set at 
0x004011E8. You can check to see if it worked by looking at the breakpoints window in 
Figure 7.21. The hotkey for the breakpoints window is ctrl-alt-B. 
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Figure 7.21 Breakpoints Window 


ll Breakpoints 
AiJdi 


P^TlI B h£frl 

Pass count HdnJwuie Curidiliun AuLiuns 


ts*t_jRfrlftfsultrirpri'.inri+A 


Rrft^k 


The new breakpoint is set to break. What we want is for it to create a log entry every 
time the breakpoint is hit. In IDA terminology, thatis a trace instead of a break. Actually, 
tracing and breaking arenit mutually exclusive.You can set it to stop execution (break) when 
it hits the breakpoint and log the event (trace) at the same time. Since weire just looking for 
which functions are executed when processing a particular message, we want all the 
breakpoints set to trace. We have to remove the break Ehg and add the trace Ehg from the 
structure. The following code adds a breakpoint at 0x004011E8 and sets it to trace. 

from idautils import * 
ea=ScreenEA() 
ea=0x004011E8 
add_bpt(ea, 1, 4) 
bp=bpt_t() 
get_bpt(ea, bp) 
bp. flags A =BPT_BRK 
bp. flags | =BPT_TRACE 

Now that we can set a tracepoint anywhere in IDA, with the list of function addresses from 
before, its a simple task to add a loop and set a tracepoint at the beginning of each function. 

from idautils import * 

funclist=[0x004011F2, 0x00401200, 0x00401206] /*add all the other 
addresses here*/ 
ea=ScreenEA() 
for i in funclist: 
ea=i 

add_bpt(ea, 1, 4) 
bp=bpt_t() 
get_bpt(ea, bp) 
bp. flags A =BPT_BRK 
bp. flags | =BPT_TRACE 
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For those cases where iris necessary to mark absolutely every function as a tracepoint, the 
IDA plug-in can be automated to Ynd every function reference instead of doing it manually. 
In practice, youill probably have a number of functions you want to eliminate off the bat, so 
the example above will be used more than the fully automated one. Just for completeness, 
hereis a python snippet that automates setting a tracepoint at every function. 

from idautils import * 

# Loop through all the functions and add a breakpoint 
for i in range(get_func_qty() ) : 
f=getn_func(i) 

print "Function %s at Ox%x" % (GetFunctionName(f.startEA) , f.startEA) 
add_bpt(f.startEA, 1, 4) 

#change all the breakpoints to trace-only 
for i in range(get_bpt_qty()): 
b=bpt_t() 
getn_bpt(i, b) 
b .flags^BPT-BRK 
b .flags | =BPT_TRACE 
update_bpt(b) 

Now that thereis a tracepoint set at the beginning of each function, simply run the client 
program to generate a hit list. 

Example Hitlist 

Small examples donit really get the point across, so a larger program is in order. Iim going to 
use Pidgin as an example (www.pidgin.im). Pidgin is a popular open-source chat program 
that supports most of the popular protocols. Since itis open source, you can compare the 
disassembly against the binary if you get lost. 

In this example, Iim going to generate a hit list against Pidgin v2.1.1 for Windows. 
Pidgin implements each protocol as a plug-in. I just picked the Yahoo! Instant Messenger 
plug-in at random. The protocol logic is implemented in libyahoo.dll. The call graph of 
the DTK is large and complex. Graphing it in IDA gives a blob that canit be interpreted 
by humans, like the graph in Figure 7.22. (Or at least by humans that ever leave the 
house.) 


Figure 7.22 "Blob" Graph from IDA 
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In this example, Iim going to mark every function as a tracepoint with the script in the 
previous section. There are 549 functions in the DLL. For the Yrst pass, Iim only interested 
in the functions used during initialization and login. The hit list should narrow the scope 
down considerably from the 549 functions to something more manageable. 

First, create an account on Yahoo! and log onto it with Pidgin to make sure everything is 
up and working. Next open libyahoo.dll in IDA and set the Debugger—>Process Options 
to run Pidgin when the debugger is launched. Finally, invoke the script from the previous 
section with alt-9 to mark every function as a tracepoint. Tell IDA to run and go get a cup 
of coffee. Running the executable under IDAis debugger takes considerably longer than 
running the executable without the debugger. When the bar at the bottom of the window 
says Available, Pidgin has Ynished logging in, as shown in Figure 7.23. 

Figure 7.23 Logging In Complete 



Now we can take a look at the trace window and determine which functions were 
involved in connecting to Yahoo!. At most weill have to reverse engineer this set of functions 
to Ygure out the login portion of the protocol so it gives us a good upper bound on the 
amount of work ahead of us. 
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The trace window (Debugger | Tracing | Trace Window) lists the functions in the 
order they were called. The breakpoint list (Debugger | Breakpoints | Breakpoint List) is 
closer to a true hit list. It lists each breakpoint and a count of the number of times that 
breakpoint was invoked, as shown in Figure 7.24. 


Figure 7.24 Invoked Breakpoints 


Si Breakpoints 

Address 

» Pass.. H 

9 textg_-iti_ha?h 

80 

| lext g_mallncO 

8 

| .textstochr 

61 

i i 

n 

| teKlpurple_account_oplion_strin. . 

6 

9 text;sub_6D98F4A0 

59 

§ te>:l'pijiple_cipher_eon(ext_digest 

540 

9 .tent;g_sti_equal 

54 

|| (extpurple_cipher_coniext_reset 

539 

9 tentstrlen 

5 

9 (sKl'purpl9_cipher_conl8xt_new 

5 

9 . (ant: g_ha£ h_ tatilajiu 9 ri 

47 

| .text;sub_SD98F2CO 

46 

9 text:purple_cmd_register 

4 

9 text:g_slist_prepend 

4 

9 t3st:sub_6D98F4C0 

38 

9 tcxt;sub_GD98F490 

33 

9 textstrtol 

3 

| l.cxi:g_!irndup 

3 

9 iextstrslr 

3 

9 .tcx(:purplc_account_gcl_string 

3 

9 .tCKi:piitplo_conncctbn_gi;l_acc... 

3 


If we eliminate all the functions that didnit get called at least once, weire down to 
133 functions. That cuts our search space down to 24% of the original. Many of the 
functions in the list are wrapper functions that simply call another function in the list. 

I like to start with the functions that are called often, but not hundreds of times. Just 
glancing at each of the functions, you should fairly quickly come to the function shown 
in Figure 7.25. 
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Figure 7.25 Frequently Called Functions 

|sub_6D9897C0 proc near 

uar_458= dword ptr -458h 
uar 454= dword ptr -4S4h 
uar_450= dword ptr -450h 
uar 44C= dword ptr -44Ch 
uar_448= dword ptr -44Bh 
ReadBuffer= dword ptr -42Ch 
oar_41C= dword ptr -41Ch 
uar 418= dword ptr -418h 
dword ptr B 


nou 

nou 

push 

push 

lea 

push 

sub 

nou 

nou 

nou 

nou 

nou 

nou 

nou 

call 

test 

nou 

jl 


edx, 40Oh 
ebp, esp 
edi 
esi 
esi, 
ebx 
esp, 
eax, 
eax, 


[ebp+uar 41B] 


44Ch ; uoid * 

[ebp+arg 0] 
[eax+ICh] 
[ebp+uar_41C], eax 
[esp+458h+uar 450], edx 
[esp■45Bh• uar_414] , esi 
eax, [eax+ *<] ' 
[esp+4b-8h+uar_4b8J, eax 
wpurple read 
eax, eax 
ebx, eax 
loc 6D989B17 


Iris always good to have dumps of the protocol in front of you for comparison. This is 

the Yrst message my client sent to the server in hex. 

59 4D 53 47 00 OF 00 00 13 00 57 00 00 00 00 00 00 00 00 31 CO 80 69 64 61 70 6C 

75 67 69 6E 31 32 33 34 35 CO 80 

The call to u>purple_read makes this function a good candidate to start reverse engineer¬ 
ing. The function copies the data into another buffer, makes sure itis longer than four bytes, 
and then drops into the main processing blocks. The Yrst protocol processing block is shown 
in Figure 7.26. 
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Figure 7.26 First Protocol Processing Block 


i 


HNuX 


loc 

AD9B9B77 

. 

cld 

nou 

esi. 

[ebp+ReadBufPer] 

non 

eax, 

offset aVnsg ; "YIISG" 

cnp 

PCX , 

PCX 

nou 

Ddl, 

eax 

repp 

cmpsb 


jz 

loc 

6D989950 



_1 1_ 


It compares the Yrst four bytes of the read buffer to the string lYMSGi and exits if it 
doesnit match. The YMSG must be used as a magic number, as shown in Figure 7.27. 


Figure 7.27 YMSG Magic Number 


pn m 

loc 60989950: 

nou 

edx, [ebp+ReacIBuffer] 

noo 

esi, offset aVahoo 0 ; "yahoo" 

nouzx 

eax, byte ptr [edx+8] 

nouzx 

edx, byte ptr [edx+9] 

nou 

[esp+458b+uar 448], ebx 

nou 

[esp+458h+SrcBuffer], esi 

sbl 

eax, a 

lea 

edi, [eax+edx] 

nou 

tesp+4'58b+l1essagel en], edi 

nou 

eax, offset aDBytesToReadRx : "Xd bytes to read, rxlen Is 3id\n" 

rtnu 

[esp+4San+Hnfferl en], eax 

nou 

[esp+458h+0estBuffer], 1 

cal 1 

purple_debug 

nou 

ecx, [ebp+uar 41C] 

1 P A 

eax, [edi+14b] 

cnp 

[ecx+BCh], eax 


short loc_6l)989M45 


The next check looks like a length Yeld. It pulls bytes 8 and 9 out of the packet, shifts 
byte 8 to the left, and adds them together. So we can guess that the length Yeld is a 
big-endian short. That leaves our structure something like: 
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struct login_packet{ 
char Magic[4]; 
char unknown1; 
char unknown2; 
char unknown3; 
char unknown4; 
unsigned short Len; 

} 

The process and number of function calls involved is greater in the Yahoo! Instant 
Messenger protocol than in the example protocol used earlier in the chapter, but extracting 
the protocol from the binary is the same. Input comes from a read and then is processed 
through a set of function calls. With a little work, you should be able to reverse the rest of 
the protocol. 


/*Always YMSG*/ 


/*Big Endian*/ 
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Introduction 

So now youlve read the book, and should be able to do some of this on your own. In this 
chapter, weire going to look at a real piece of hostile code. The hostile code weire going to 
use is real, this means that you should be especially careful when dealing with it yourself 
because you could possibly do serious harm to your computer and your network. Please be 
positive you are authorized to analyze the application in your environment. I highly suggest 
the use of some form of virtualization software, such as VMware. One thing I especially like 
about VMware is the ability to take snapshots, which allows me to get to speciYed points and 
make a restore point. This makes things like travel and close of the business day easier as you 
donit have to worry about someone unplugging your computer or needing to take your 
laptop home with you. All that said, be careful and bear in mind that you and only you are 
responsible for your actions. In that sense, when we say advanced walkthrough, we donit 
mean so much that the content is especially advanced, but rather that weire taking all of the 
pieces and putting them together to form a coherent picture of what you can expect to be 
doing as a reverse engineer. 


Note 

Just like anything else, reverse engineering is something you will get more 
comfortable as you gain more experience doing it. Thus, once the base 
knowledge is there, it's really just a matter of doing it to become good. 

It can be problematic trying to find employment in the field until you're 
more qualified to do it. That's where the beauty of the internet comes in. 
There are many organizations and Web sites that have challenges, for 
instance the Honey Net project is fairly well know for presenting incident 
response type challenges. This can be fun and is a good way to get your 
notoriety levels increased, however the focus is rarely on reverse engineering. 

There are a couple of especially good websites. The first and most 
obvious is http://crackmes.de. This website is full of user-submitted programs 
with various objectives for reversing. For instance, they have unpackme's 
where the challenge is to unpack the program; crackme's which simulates 
cracking commercial software and so on. They have challenges for multiple 
operating systems and have various degrees of difficulty. 

The second Web site is more of a forum, and there are some challenges 
but its one of the best places to find likeminded people, http://community. 
reverse-engineering.net/. This is a forum that has a plethora of information 
and will give you access to smart people. 

The third is the Offensive Computing website http://www.offensive- 
computing.com/, this Web site among other things maintains a repository 
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of known malware and serves as a good site for cross-referencing anything 
you might find, and a good place to brush up on your reversing malware 
skills using the real deal. 

Finally, there is another Web site known as OpenRCE, http://www.openrce. 
org, it's run by a fairly accomplished reverse engineer from TippingPoint 
named Pedram Amini, and it contains forums and plugins, scripts and so on. 
There also is a searchable listing of call chains for the main system libraries, 
which comes in handy more often than you think. 


Reversing Malware 

As I said earlier, this program really isnit malware per se but rather ad-ware (itis at least 
somewhat safer that way), but it is a real program that you can expect to Ynd in the wild, 
especially if you work on an incident response team or similar. Furthermore, many people 
would include usage of tools like regmon, Ylemon and so on, and these applications have 
valid use especially in the business world where itis always a race against time. However, 
thatis not reverse engineering, weire engineers not fortune tellers and if given the time 
there is really no reason to have to do this, you know what the program is doing while it 
does it, so there should rarely be a technical reason to have to use these tools, unless youire 
not actually reversing the binary. The other reason for this is that doing things like that is 
borderline dangerous, you let the program have control and you really canit know what it 
did, there is nothing that says you need to touch the Mesystem, or registry, or standard API. 
Ultimately, if given the right application these methods will fail, plus this is a reverse 
engineering book. That said, you should probably know how to use these tools as there will 
be any number of times where the time allocated by management just wonit allow you to 
do the job correctly-time is money. Without further ado though, letis just right into the 
application and start looking at it.You can download this code from www.syngress.com/ 
solutions. The password to open the Yle (courtesy of Dan Kaminsky) is elDANGER!- 
INFECTEDMALWARE!DANGER!i. 

Now as we examine the beginning of this code, itis pretty obvious that this is a 
non-standard entry, this is usually a sign of self-modifying code in some form or fashion, 
often its evidence of a packer. We have at least three tell-tale hints in this section. The Yrst 
and most obvious is the name of the section weire in is UPX1, which is standard for a 
program packed with UPX. The next is the hexadecimal data preceding our entry point. 
This is pretty typical for both packed and encrypted code/strings/et cetera, and Ynally the 
last signoand this one takes a little bit (but not a lot) of experience, is the pusha instruction. 
UPX is trivial to bypass as it saves all the registers with a pusha, then restores them with a 
popa followed by an unconditional jump. Thus itis just a matter of breaking at the popa/jmp 
and then continuing from there or dumping the data to disk. We really should conYrm that 
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what we expect is whatis actually going on.We can do this by examining the code and/or 
single-stepping through it, and then conYrming that the popa/jmp combination is actually 
present. You would normally do the Yrst by examining control Ebw; dynamically if necessary. 
Weire not going to do that here because there is really no point in including 15 pages of me 
single stepping through UPX unpacking an application to conYrm that control doesnit jump 
off into someplace weird. I wouldnit recommend doing that in general, and overall I would 
highly advise doing analysis inVMware or something similar. 

Figure 8.1 popa/jmp Instruction 


UPX1:0040A06fl loc_40806fl: 

* UPX1:0040A06fl mou 

* UPX1:0040006F push 

* UPX1:00400070 call 

' UPX1:0040A076 or 

UPX1:0040AO78 jz 

' • UPX1:0040A07A mou 

I * UPX1:0040A07C add 

—UPX1:O040A07F jmp 

I UPX1:004OA081 ; - 

i UPX1:004 0A 081 
| UPX1:O040A081 loc_4OA081: 

+* UPX1:0040A081 call 

UPX1:0040A087 
UPX1:0040A087 1OC_40A087: 

-►* UPX1:0040A087 popa 

—? UPX1:00400088 jmp 

UPX1:00400088 ; - 


; CODE XREF: UPX1:0040A062tj 

ecx, 0AEF248S7h 
ebp 

dword ptr [esi+0O484h] 
eax, eax 
short loc_40A081 
[ebx], eax 
ebx, 4 

short loc 400059 


; CODE XREF: UPX1:0040O078tj 
dword ptr [esi*0A488h] 

; CODE XREF: UPX1:0040O040tj 

near ptr word_4 030 5E 


As we can see in Figure 8.1, what we expected is there exactly- at loc_40A081.We have 
the popa/jmp instruction. Typically, in order to use IDAis unpacker plug-in we would need the 
Original Entry Point (OEP).However, because UPX wasnit exactly made to be obfuscated/ 
used for malicious software, the OEP is obvious and is word_40395E, lets just take a quick 
look at that address and just make sure this is an address that makes sense. (See Figure 8.2). 

Figure 8.2 Examining the Address 


UPX1:00408000 
UPX1:00408 000 
UPX1:00408000 
UPX1:00408 000 
UPX1:004O8000 
UPX1 -.00408000 
UPX1:00408000 
UPX1:00408000 
UPX1:004 08000 
UPX1 .-00408 000 
UPX1:00408 000 
UPX1:00408000 
UPX1: 00408000 
UPX1:00408000 
UPX1:0O408000 
UPX1:00408000 
UPX1:00408000 
UPX1:00408000 


dd 0AF4BF3Ch, 
dd 15E16F1Bh, 
dd 0F2C0337Fh 
dd 0C14FCO8Bh 
dd 272CF72Ah, 
dd 7C2CC120h, 
dd 9CEEFB5Bh, 
dd 0FCDED982h 
dd 8E1B0D0Ch, 
dd 70C16BBEh, 
dd 0BBCCEO38h 
dd 20868EEh, 
dd 5118041Fh, 
dd 6468DDB9h, 
dd 815B5D1Bh, 
dd 7E1274EEh, 
dd 35CE3S5Ch, 
dd 0E5D0EOEh, 


59BF2E05h, 0F72CB9FBh, 37400189b, 0FBFF5060h 
5953426Ch, 0C98309A5h, 6028BFFFh, 0B7BF2B0Eh 
, 28D1F70Eh, 0F78BD1FEh, 11635813h, 7FDBF7CCh 
, 0A5F302E9h, 3E18307h, 242C04F3h, 0C1B58B99h 
0B2OF2O16h, 20BF4F60h, 582A1860h, 7746B363h 
3E9B2141h , 5FC8918Ch, 1CEC8111h, 0F68B530Eh 
55072824h, 40242D8Bh, 0DB85C43Ah, 17C3840Fh 
, 24B488B0h, 5303603011, 0O268E456h, 0C2B67B9Bb 
8DE562FBh, 0E198 0D82I1, 0C2182494b, 9212FA6Bh 
0C0507222h, 590FB970h, 0BDC2FO1Bh, 1C24848Dh 
, 8C8B2F9Fh, 51933824b, 532015FFh, BFDBCIOBEb 
0FFE92604h, 92B48DD5h, 16C7D9DDh, 8C8D20EBh 
16CD7756h, 45C207ECh, 80B73D1Ch, 0B35B2O73h 
458688Ah, 6501816h, 6685F058h, 0D005E62Bh 
9B3982C4h, 0CC216FDh, 5256CO00b, 0D8397718h 
50130528b, 7C10EB530, 3E3B3305h, 2C55111h 
0A7289B9Fh, 7150029Ch, 63962CCF0, 10485884h 
8316CD87h, 254CBF38h, 9EE0DC58h, 682B1EF0h 


k 
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Looking at this address, it points into an offset into the long sequence of hexadecimal 
characters, which is exactly what we expect as the entry point doesnit typically point to the 
very beginning of the original Me. So having a reasonable assurance that what weire looking 
at actually is UPX, lets see if we can get it unpacked. All recent versions of IDA (since 4.8) 
include a plug-in that is supposed to perform universal unpacking. Weire going to brieEy 
walk through that just in case youire not familiar with it. However, just like the previous 
chapter weire not going to get into unpacking too much. 


Figure 8.3 Universal PE Unpacker 



MkN 


ill File Edit Jump Search View Debugger Options Windows Help 

H : D ;p Ctr'-bs I 1 | Text_ ^ 


1% - 

Clrl-Ins 

Begin selection 
! Select identifier 

Alt-L 

Shift- Enter 

1 1 

1 coo Code 

C 

J dat Data 

D 

ftC Struct var... 

Ej "s" Strings 
* Array... 

Alt+Q 

► 

Num * ! 

X Undefine 

N Rename 

U 

N 

1 

Operand type 

► 

Comments 

► 

Segments 

► 

Structs 

► 

Functions 

► 

Other 

► 


m N TBi^ T 


Imports | N Names" % t£l) Functions Strings | ft Structures 
;org 408000h 

assume es:nothing, ss:nothing, ds:UPX 
00 dd 0FCDE6FFCh, 0C7F18B56h, 46C70006h, 

; DATA XREF: 

dd 90C35EC6h, 9B9FFD8Bh, 84CC121Fh, 8 
dd 25FFF7h, 8168FF6Ah, 6400403Bh, 645 
dd 51FFBBFFh, 0F98B5756h, 262D406flh, 
dd 0F6850824h, 142444C7h, 0F6F7DBBAh, 
dd 114C614h, 0C19E7FB7h, 0B802280Ch, 
dd 75FFDB3Bh, 0C0325F13h, 244C8B5Eh, 

_-* -* nr- n ««-_iM-i-np 


► Universal PE unpacker 


Alt-1 


12h , 667C9E7Bh, 
Ih, 1O8B0894h, 4 
Ih, 0D5O49C85h, 
>h, 2CAB08EDh, 8 
h, 51F6FDAEh, 5 
»h, 3B41044Dh, 8 
ih, 2D1F7412h, 5 
i8h, 59124689h, 

il ni\rnmrr\/L 


UPX1:00408000 
UPX1:00408000 
UPX1:00408000 
UPX1:00408000 
UPX1:0040800O 
UPX1:00408000 
UPX1:00408000 


Sample plugin 

Jump to next Fixup 

COM Helper 

Sample chart builder 

Change the callee address 

Borland RTTI 

Reoeat RTTI command 


Alt+5 

Alt+Fll 

Alt+F9 

Alt+F5 


Alt-0 
Alt- F12 


From inside of IDA, getting to the plug-in is pretty simple, it can be found under Edit > 
Plugins > Universal PE unpacker or you can simply hit Alt+1 (see Figure 8.3). 

From there, a warning letting you know that if youire not careful you could own yourself 
is displayed, and the dialog seen in Figure 8.4 appears. 
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As you can see, knowing the exact OEP is not entirely necessary. Basically, this technology 
is not very advanced in the sense that the debugger doesnit know what its doing. All it does is 
watch for execution in the range including/between Start Address and End Address. In this 
case, I modiYed the default End address parameter to reEfect our binary and then clicked OK. 
When IDA detects that the necessary conditions have been met, it prompts you with the 
dialog shown in Figure 8.5. 

Figure 8.5 Confirm Dialog 



Once OK is clicked, IDA terminates the debugger and does a lot of stuff in the background 
for you. For example it rebuilds the imports, reanalyzes code Ebw, et cetera. Once all of that is 
complete weire looking at the unpacked version of the code, which is a bit closer to what we 
expect to see (Figure 8.6). 
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Figure 8.6 Unpacked Version of Code 


H N Ui 


; Attributes: bp-based frame 

public 

start 

start proc near 

var 78= 

dword ptr -78h 

uar 74= 

dword ptr -74h 

uar 78= 

dword ptr -78h 

uar 6C= 

dword ptr -6Ch 

uar 68= 

duord ptr -68h 

uar 64= 

duord ptr -64h 

uar 68= 

duord ptr -6Oh 

StartupInfo= STARTUPINFOA ptr -5Ch 

uar 18= 

duord ptr -18h 

uar 14= 

duord ptr -14h 

uar_4= 

duord ptr -4 

push 

ebp 

nou 

ebp, esp 

push 

OFFFFFFFFh 

push 

offset unk 404538 

push 

offset except handler3 

nou 

eax, large fs:0 

push 

eax 

nou 

large fs:0, esp 

sub 

esp, 68h 

push 

ebx 

push 

esi 

push 

edi 

nou 

[ebp+uar_18], esp 

xor 

ebx, ebx 

nou 

[ebp+uar 4], ebx 

push 

2 

call 

_set_app_type 

pop 

ecx 

or 

duord ptr unk 406454, OFFFFFFFFh 

or 

duord ptr unk 406458, OFFFFFFFFh 

call 

p fnode 

nou 

ecx, dword ptr unk 406448 

nou 

[eax], ecx 

call 

_p_commode 

nou 

ecx, dword ptr unk 406444 

nou 

[eax], ecx 

nou 

eax, adjustfdiu 

nou 

eax, [eax] 

nou 

dword ptr unk 406450, eax 

call 

nullsub 1 

cnp 

duord ptr unk 4062A0, ebx 

jnz 

Short loc 4039E1 


HNlU _ 


eax, [ebp*uar_14] 
ecx, [eax] 
ecx, [ecx] 
[ebp*uar_78], ecx 
h eax 

h ecx 

1 _XcptFilter 

ecx 
ecx 
n 


Examining the code, we see a typical routine entry, however if you note references to 
routines like nullsub_l, and if you were to dig a bit deeper youid Ynd references like those 
shown in Figure 8.7. 
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.mat a 

0O4B41 /<t 

extrn 

_imp 

_y 41:owora 

PH 1 H 

AKtr 

i r 

> 

.idata 

00404178 

extrn 

_inp! 

_1576:duord 

DATA 

XREF 

1576Tr 

» 

.idata 

0040417C 

extrn 

_imp 

_2976:duord 

DATA 

XREF 

_ 2976tr 

» 

.idata 

00404180 

extrn 

_imp 

_3081 rdirord 

DATA 

XREF 

308lTr 

• 

.idata 

00404184 

extrn 

_imp 

_2985:dword 

DATA 

XREF 

298ST r 

1 

.idata 

00404188 

extrn 

_imp 

_3262:duord 

OAIA 

XREF 

3262tr 

• 

.idata 

0040418C 

extrn 

_imp 

_3136:dword 

DATA 

XREF 

3136Tr 

• 

.idata 

00404190 

extrn 

_imp 

_1776:dword 

DATA 

XREF 

1776Tr 

• 

.idata 

00404194 

extrn 

_imp 

_101_1: duord 

; DATA XREF: 101 Itr 

> 

.idata 

00404198 

extrn 

_imp 

_101_2:dword 

; DATA XREF: 101 2Tr 

• 

.idata 

0040419C 

extrn 

_imp 

_1#1_3:dword 

; DATA XREF: 181 3t|- 

» 

.idata 

00404100 

extrn 

_imp 

_5714:duord 

DATA 

XREF 

5714T r 

» 

.idata 

00404104 

extrn 

_imp 

_5289:duord 

DATA 

XREF 

5289tr 

» 

.idata 

00404108 

extrn 

_imp. 

_5307:duord 

DATA 

XREF 

_ 5307Tr 

» 

.idata 

0040410C 

extrn 

_imp 

_4698:dword 

DATA 

XREF 

4698tr 

• 

.idata 

004 041B0 

extrn 

_imp 

_4079:dword 

DATA 

XREF 

4079tr 

» 

.idata 

00404164 

extrn 

imp 

_2725:dword 

DATA 

XREF 

2725T r 

> 

.idata 

004041B8 

extrn 

_i«p] 

_5302:dword 

DATA 

XREF 

5302tr 

» 

.idata 

004041BC 

extrn 

imp 

_5300:duord 

DATA 

XREF 

53O0Tr 

• 

.idata 

004041C0 

extrn 

_imp 

_3346:duord 

DATA 

XREF 

3346tr 

1 

.idata 

004041C4 

extrn 

_imp 

_2396:dword 

DATA 

XREF 

2396t|- 

» 

.idata 

004041C8 

extrn 

_imp 

_5199:duord 

DATA 

XREF 

5199tr 

• 

.idata 

0O4O41CC 

extrn 

_imp 

_1089:d«»ord 

DATA 

XREF 

1089tr 

• 

.idata 

00404100 

extrn 

_imp 

3922:duord 

DATA 

XREF 

3922T r 

• 

.idata 

004041D4 

extrn 

_.imp 

5731:dword 

DATA 

XREF 

573lTr 

» 

.idata 

00404108 

extrn 

_imp. 

_2512:dword 

DATA 

XREF 

2512tr 

» 

.idata 

004041 DC 

extrn 

_imp 

_2554:doord 

DATA 

XREF 

2554tr 

• 

.idata 

004041E0 

extrn 

_imp 

_4486:duord 

DATA 

XREF 

4486tr 

» 

.idata 

004041E4 

extrn 

imp 

6375:duord 

DATA 

XREF 

637Slr 


As you can see, IDA has not reanalyzed all of the imported functions. This happened 
because the FLIRT signatures that IDA typically runs over the code were not applied. This is 
not a huge issue because we can do it ourselves without much effort. If youire unfamiliar with 
this, thatB okay, as weire going to talk about it some. 


Figure 8.8 Load File 


Hi i File Edit Jump 

G New... 

1 Open... 


Search View Debugger Options Windows Help 


}76 - 


BE 


| Produce file 
|j © IDC file... 

© IDC command... 
y Save 
[¥ Save as... 

Close 


Shift* F2 
Ctrl+W 


Reload the input file 
► Additional binary file... 
IDS file... 

PDBfile... 

DBG file... 

TDSfile... 


9. — 

0 . 


XI Exit 


Alt+X 


.idata:00404168 


extrn _imp_5861:dword 






Parse C header file... 

Ctrl*- F9 

id 

lThTR 

XKLT 

i r 


rd 

DflTft 

XREF 

924tr 


ord 

DBTfi 

XREF 

2446tr 


rd 

DfiTfi 

XREF 

850tr 


rd 

DATA 

XREF 

926tr 


lord 

DATA 

XREF 

6883tr 


ord 

DATA 

XREF 

6663tr 

rd 

DATA 

XREF 

861tr 

rd 

DATA 

XREF 

539tr 


ord 

DATA 

XREF 

160lTr 


ord 

DATA 

XREF 

1572lr 


ord 

DATA 

XREF 

_6143tr 


ord 

DATA 

XREF 

2645tr 

. 

rd 

DATA 

XREF 

537Tr 

di'jord 

DATA 

XREF 

5861tr 


ww.syngress.com 











Advanced Walkthrough • Chapter 8 173 


In order to get to the options for loading FLIRT signature YLes, you go to File > Load 
File > FLIRT signature file (Figure 8.8), from this menu you get the prompt in Figure 8.9. 


Figure 8.9 List of Available Library Modules 


r " 

J .p List of available library modules 


File Op.. 

Library name 


Borland Visual Component Ubiary & Packages 

.flb5132mfc 

Borland 5.Ox MFC adaptation 

-|ib516cgw 

BCC v4.5/v5.x CodeGuard 16 bit 

<^b532cgw 

BCC v4.5/v8x CodeGuard 32 bi» 

■^bc15bids 

BCC++ for OS/2 dasslib 

>Jlbc15c2 

BCC++ foi OS/2 runtime 

■*Pbc15owl 

BCC++ for OS/2 OWL 

«jibc31cls 

TCC++/BCC+* dasslib 

«^lbc31owlw 

BCC++ v3.1 OWL 

-Jfc bc31 rtd 

TCC/TCC**/ i BCC 4-+ 16 bit DOS 

>^bc31ttw 

BCC++ v3.1 windows runtime 

bc31 tvd 

TCC++/BCC++ TVision 

^Ibcb5rf 

CBuilder 5 runtime 

■»$&bdsexl 

BDS v8 Extra (lechology) hbrary 

■^bhlb'ds 

BCC v4.x/b.x class library 16 tut 

^bh16dbe 

Borland DBE 16 bit 

^ bhl Bgrfd 

BCC v4.x/5.x BGI graphics 

«^bh16ocf 

Borland OCF16 bit 

-Jlbh16owl 

Borland OWL (2/2.5) 16 bit 

■^bh16rdos 

BCC v4.x/5.xD0S runtime 

■>Jibh16fwin 

BCC v4.x/5.x windows runtime 

-flbh32ds 

BCC v4.x/5.x dass library 32 bi 

-jibh32dbe 

Borland DBE 32 bit 

^»bh32ocf 

Borland OCF 32 bit 

■»Jibh32owl 

Borland OWL (2/2.5) 32 bit 

^bh32rw32 

BCC v4.x/5.x i BCB vl 0/v7.0 BDS2006 vwi32 runtime 

<£1 b«S_31 

BSDI386 3.1 kernel 

c4vcl 

CBuilder 4 and Delphi 4 VCL 

-Jicsetadd 

IBM C Set v2x/3.x class library 

cselrto 

IBM C Set v2x/3.x 

>^ctask 

CTask libraiy 

%Jld3vcl 

Delphi 3 Visual Component Library 

■jid4vcl 

Delphi 4 Visual Component Libraiy 

>Jld5vcl 

Delphi 5 Visual Component Library 

N^delphi 

Delphi VI 0 

>jfcetnx32add 

EMX additional libraries 

>Jlemx32rto 

EMX runtime 

-^exe 

Startups of CXC/COM fHei 

gcc32rtf 

GCC (mingw/cygwin) v3.4 runUme 

■^hfgh/to 

Metaware High C for OS/2 

^icknat 

Intel C specie tbraryes 

^latrtd 

Lattice C v3.30 


OK ] [ Cancel | Help | j Search 


Line 1 of 124 


Looking at the prompt, we get some idea of what it does; speciYcally we see a list of 
compilers/libraries. Thus the FLIRT signature Me applies characteristics known to exist for 
a given compiler. But how do we know what compiler is in use? This at Yrst seems like it 
would be implausibly hard, but generally itis pretty easy and failing that itis where some 
experience comes in. Usually, we can look through the strings in a given binary and the 
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compiler inserted a string advertising its use. Failing that we can look at the generated code 
and try to guess. What to look for exactly when examining the code is a bit beyond the 
scope of this book.Youill get a better idea of how to do that as time progresses. For instance 
however, if youive looked at much code generated by GCC you will probably note that it 
likes to allocate more space on the stack than necessary, then in the next instruction or two 
it will correct that, youire basically looking for things like that. In our instance, these are our 
strings shown in Figure 8.10. 


Figure 8.10 Strings 


J2 IDA View-A [ fel Hex View-A ] 

rO) Exports j Imports | N Names j Functions ’ " Strings ^ £ 

Address 

Length 

type 

String 

" " UPX0:0.. 

OOOOOOOC 

c 

wininit.ini 

UFXOiO... 

00000008 

c 

%s=%s\r\n 

UPXOiO.. 

00000009 

c 

Kernel32 

" " UF’XOiO... 

OOOOOOOC 

c 

MoveFileExA 

“•••" UPXOiO.. 

00000005 

c 

Imp 

"• " UF’XOiO. 

OOOOOOOC 

c 

Ad_AdSen_20 

" " UPXOiO.. 

00000017 

c 

SOFTWAREWMexa Toolbar 

" " UPXOiO 

00000017 

c 

R egisterS erviceProcess 

" ■" UPXOiO.. 

00000009 

c 

KERNEL32 

" " UPXOiO... 

0000000B 

c 

\\AdsNT.exe 

UPXOiO.. 

00000006 

c 

AdsNT 

" " UPXOiO... 

00000008 

c 

Version 

UPXOiO.. 

0000000B 

c 

\\AdsNT.ini 

UFXOiO... 

0000000B 

c 

Windex.htm 

" " UPXOiO.. 

0000000A 

c 

Referer: 

UPXOiO... 

00000006 

c 

adsnt 

" " UPXOiO.. 

00000040 

c 

S 0 FT WAR E WM icrosof t\\Windows\SCurrentVersion\\policies\\E xplorerWR.. 

" " UPXOiO... 

0000000B 

c 

adsntB/1.6 

UPXOiO.. 

0000000E 

c 

AdsNTGroupURL 

"• " UFXOiO... 

00000009 

c 

gutiZdZd 

UF’XOiO.. 

00000008 

c 

grouped 

" ..." UPXOiO 

OOOOOOOC 

c 

showS tyle^d 

" " UFXOiO.. 

0000000F 

c 

showlEWindow^d 

" " UFXOiO... 

00000009 

c 

weighted 

" " UPXOiO.. 

00000009 

c 

objur&d 

" " UFXOiO... 

00000006 

c 

url^d 

' •• UFXOiO.. 

00000009 

c 

height^d 

" UFXOiO.. 

00000008 

c 

width^d 

" "UFXOiO.. 

00000009 

c 

toppos^d 

11 " UFXOiO... 

00000009 

c 

AdsNTURL 

" " UPXOiO.. 

000000CA 

c 

leftpos^d 

" " UFXOiO... 

00000006 

c 

AdNum 

" " .newllD:. 

0000000D 

c 

ADVAPI32.dll 

" " .newllD:.. 

0000000F 

c 

RegSetValueExA 

" .newllD:. 

00000011 

c 

R egQ ueryValueE xA 

" " .newllD:.. 

0000000E 

c 

RegOpenKeyExA 

.newllD:.. 

OOOOOOOC 

c 

RegCloseKey 

" " .newllD:.. 

00000010 

c 

RegCreateKeyExA 

1 " newllD:. 

0000000D 

c 

kernel32.dll 

" " .newllD:.. 

00000011 

c 

GetModuleHandleA 

" " .newllD:. 

0000000B 

c 

GetVersion 

" " newllD:.. 

000000QA 

c 

CopyFileA 

*-* newllD;. 

00000012 

c 

G etS hortPathN ameA 

" " newllD:.. 

0000000D 

c 

ReleaseMutex 

" " newllD:. 

0000000D 

c 

GetLastError 

" " newllD:.. 

0000000D 

c 

CreateMutexA 

" ■ newllD:. 

00000014 

c 

GetCurrentProcessId 
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Unfortunately, we donit get to see any compiler strings so we have to take an educated 
guess. If worse comes to worse we can just start loading FLIRT signatures until weive resolved 
most everything. In our instance, this is mostly what happened. There are numerous signatures 
that matched against. The Yrst and most common is the MS Visual C++ runtime signature, 
but then using MFC starter signatures turned up the most results. In the end, I managed to 
get most everything resolved, this particular binary was a little wonky, but thatis entirely because 
itB using MFC 4.2. With all of that resolved, towards the end of the entry points function, 
we see that the call at the end has been resolved to aWinMainQ routine, and when we look 
at that we see Figure 8.11. 


Figure 8.11 WinMainQ Routine 


ih n i.g 


; Attributes: library Function 

; int _stdcall WinMain(HINSTANCE hlnstance,HINSTANCE hPreuInstance,LPSTR lpCndLine,int nShowCnd) 

_WinMain@16 proc near 

hlnstance= dword ptr 4 
hPreuInstance= dword ptr 8 
lpCmdLine= dword ptr 0Ch 
nShowCmd= dword ptr 10h 

push [esp+nShowCnd] 

push [esp+4+lpCmdLine] 

push [esp+8+hPreuInstance] 

push [esp+OCh+hlnstance] 

call ?AfxWinHain0@VGHPAUHINSTANCE_0@0PADH@Z ; AfxWinMain(HIMSTANCE_ ,IUMSTANCE_ -,char r int) 

retn 10h 

_WinMain@16 endp 


The one interesting thing is that the code is using MFC. I believe out of all the malware 
live analyzed, this is only the second or third that employs MFC. Now, live managed to spend 
a fairly signiYcant portion of my life without learning much of anything about MFC, and if 
weire lucky we can continue that trend today. So I set a breakpoint on the call and single 
stepped into the call, hoping that I wouldnit hit a ton of MFC kludge (See Figure 8.12). 
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Figure 8.12 General Registers 


nfc42.dll:72A8D4F3 mfc42_1576: 

' nfc42.dll:72A8D4F3 nou edi, edi 

* nfc42.dll:72A8D4F5 push ebp 

* nfc42.dll:72A8D4F6 nou ebp, esp 

* nf c42.dll:72A8D4F8 push ebx 

' nfc42.dll:72A8D4F9 push esi 

* nfc42.dll:72A8D4FA push edi 

* nfc42.dll:72A8D4FB or ebx, BFFFFFFFFh 

* nfc42.dll:72A8D4FE call near ptr mFc42_1175 

* rafc42.dll:72A8D503 nou esi, eax 

* nFc42.dll:72A8D505 call near ptr unk_72A7EE21 

* nfc42.dll:72A8D50A nou ecx, [ebp+l0b] 

* nFc42.dll:72A8D5BD nou edx, [ebp+0Ch] 

’ nfc42.dll:72A8D510 nou edi, eax 

' nfc42.dll:72A8D512 nou eax, [ebp»14h] 

' nfc42.dll:72A8D515 push eax 

* nfc42.dll:72fl8D516 nou eax, [ebp-*8] 

* nFc42.dll:72A8D519 push ecx 

* mfc42.dll:72A8D51A push edx 

* nFc42.dll:7288D51B push eax 

* raFc42.dll:72B8D51C call near ptr nFc42_1575 

' nFc42.dll:72B8D521 test eax, eax 




nfc42.dll:72A8D523 jz short 10C_72B8D555 

nfc42.dll:72A8D525 test edi, edi ~ 

nfc42.dll:72A8D527 jz short loc_7288D539 

mfc42.dll:72A8D529 nou edx, [edi] 

nFc42.dll:72fi8D52B nou eax, [edx*8Ch] 

nFc42.dll:72fi8D531 nou ecx, edi 

nfc42.dll:72A8D533 call eax 

nFc42.dll:72B8D535 test eax, eax 

nFc42.dll :72A8D537 jz short loc_72A8D555 

nfc42.dll:72fi8D539 
nfc42.dll:72A8D539 loc_72A8D539: 
nfc42.dll:72A8D539 nou 
nfc42.dll:72A8D53B nou 
nfc42.dll: 72A8D53E nou 
■■■■72A8D540BB 
nfc42.dll:72A8D542 test 
nfc42.dll:72A80544 jz 
nfc42.dll:72A8D54A nou 
nfc42.dll:72A8D54C nou 


♦ General registers 


EAX |00401800 U|sub_4O180O 
EBX'FFFFFFFF u| 

ECXJ 00406360 l + JdPX0:dword_406360 
EDX 00404338 l + |uPX0:off_404338 
ESI 00406360 l*|uPX0:dword_40636 0 
EDI | 00406360 lJuFf<0:dword_406360 


0012FEE4 U»|Stacit[ 000OOEE 4 ]: 0012FEE4 

72A8D54O4|nfc42.dll:72A8D540 

00000202 


loc_72A9BE75 
edx, [esi] 
eax, [edx+SCh] 


This was a fairly simple function and getting to what we were looking for was rather 
painless. Because this is malware live never analyzed and Km not entirely positive what it 
does, nor the mechanics of AfxWinMainQ, I stepped over the internal calls to other MFC 
APIs. I had noticed the two calls to the EAX register and Ygured that one of them was 
probably what I was looking for. The second one was, as weive noted it hands control back 
into the application to sub_401800. Now weire going to terminate the debugger and continue 
static analysis at sub_401800. After terminating the debugger, I simply went to the functions 
tab, which can be found in between the strings and names tabsoif itis not there you may 
have closed it and you can reopen it by going to View > Open Subviews > Functions, 
or hitting Shift+F3. Once there we see the code in Figure 8.13. 


www.syngress.com 



Advanced Walkthrough • Chapter 8 177 


Figure 8.13 


SUN id 1 

sub_4O1800 proc near 


uar D2» 

byte ptr -0D2h 


uar 00= 

dword ptr -00Oh 


uar C= 

duord ptr -0Ch 


uar_4= 

dword ptr -4 


mou 

eax, large fs:0 


push 

OFFFFFFFFh 


push 

oFfset loc 403BBB 


push 

eax 


nou 

large Fs:0, esp 


sub 

esp, 0C8h 


push 

ebx 


push 

esi 


push 

edi 


push 

oFFset Name ; 

"fid fldSen 20" 

push 

1 ; 

blnitialOwner 

nog 

edi, ecx 


push 

o ; 

lpllutexfittributes 

call 

CreateMutexfi 


nou 

esi, eax 


call 

GetLastError 


cnp 

eax, 0B7h 


nou 

bl, 1 



short loc_401844 



HNul 



HNld _ 

loc_401844: 

test esi, esi 

jz short loc_40184F 



_t_ 


[H N Ld 


push 

call 

esi 

ReleaseMutex 

; hMutex 



As we can see, the subroutine takes no arguments, and has relatively few arguments. 

The prologue of the procedure is pretty standard. The Yrst real code we see is a call to 
CreateMutexA() with an argument of iAd_AdSen_20i.This is potentially used to synchronize 
access between threads, and we will know for sure as we continue into the application. 
However, itk quite likely that this mutex actually prevents double-infection and from multiple 
copies of the application from trampling itself. Another aspect is that if we Googled for that 
string, weid probably Ynd what information we were looking for and we basically just acted 
as a human anti-virus application. However, we will just assume that we didnit get any results 
and continue analyzing. 
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What we see here after the call to CreateMutexA() is that the return value is copied into 
the ESI register and GetLastErrorQ is called. This return value is compared against the constant 
value 0xB7, or 183, which is the value for ERROR_ALREADY_EXISTS.This is exactly 
what weire expecting to see if theyire trying to prevent themselves from trampling themselves. 
From there, the value of 1 is copied into the lower 8-bits of the EBX register, and then we 
branch depending on the return value. If the condition was true, that CreateMutexA() failed, 
then we branch to loc_401844, otherwise we copy a value from the stack into the low-order 
8-bits of EBX. From there, we test to see if the ESI register is non- zero (CreateMutexQ 
returns NULL on error). If this is the case we branch off to loc_40184F, otherwise we release 
ownership of the mutex. Moving on down the function we Ynd the code in Figure 8.14. 

Figure 8.14 



Moving on through the code, we see the potential call to ReleaseMutex(). Following that, 
at loc_40184F the low-order 8-bits of the EBX register is checked against the value of 1. 

If you remember correctly this constant value was moved in and remained in there if the call 
to CreateMutexAQ failed, thus the code at loc_40184F simply checks to see if the mutex 
was successfully created/ownership was giving to the calling thread. If the call failed, execution 
control is handed off to loc_40188E, otherwise it continues down. Assuming there was no failure. 
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then zero is pushed onto the stack as a parameter to sub_4018B0().We also note that the 
address of an offset in var_D0 is placed in the ECX register immediately before calling the 
routine, and then before itis used again the value is overwritten with a different offset. There are 
two possible explanations here, the Yrst is that a fastcall calling convention is being employed, 
and the second is that we are calling a method inside a class. Because the Yrst argument is 
pushed on the stack, instead of being placed in the ECX register (with the second being in the 
EDX register), we can assume that we are calling an instance method and that what exists in 
the ECX register is the this pointer.Weill now take a look at sub_4018B0() and see what it 
does and try to conYrm our suspicions about the ECX register shown in Figure 8.15. 


Figure 8.15 ECX Register 


sub 4018B0 : 
push OFFFFFFFFh 

push offset loc 403C3C 

mov 03x, large fc:0 

push eax 

mov large fs:U, esp 

push ULX 

mnv eax, [es-p+lllh+arg_ll] 

push esi 

push yd i 

mnv esi, erx 

puoh 03 x 

push GGh 

mov [espCZUhi var 10] r esi 

uj. 11 ?7DCDialuy8SQAE©IPAVCVud«8fiZ; CD ialuy : :CDi aluy(uinl,CVuil *) 
lea ecx, Lesi-»-G0hJ 

mov [esp»18hivar 4], Q 

call 770C5tring@£QAC@XZ; CString : :CString(void) 

lea ecx, [esi+Mh] 

mov byto ptr rocp+16h+vor_4l, 1 

call ??0C3li iny8@QAE@XZ; C3 t» i ny ; ;CSlr iiry( vu id) 

lea ecx, [esi*68h] 

muv by Lu pli [ysp+lBh+v«n _4] , 2 

call sub_401000 

lea ecx, [esi«78h] 

muv by Lw pit [ysp+18li+var_4], 3 

ra I I v /||i:str ingArrayP.dlJAFKX/; CStringArray : :t;st.ringArray(vnirl) 

loa qcx, roci+90hl 

muv by Lw pli [ysp+18li+vai _4] , 4 

cal 1 77 QCStringArrayei?QAEeXZ; CString Array : :CStringArrayC void) 

loa ocx, rcci^0A4hl 

mov byte ptr [esp+lGh+var_4J r 5 

call 7 70CString@@QAE@XZ; CString : :C5tring(void) 

loa etx, [esi*0A8h] 

mnv hyte ptr [esp+1Hh+var_4], h 

call 77 OCSlringArray@i*QAE@XZ; CString Array : :CStringArray( void) 

lea ed i , [eti*0BCh] 

mov byte ptr [espil8nivar 4], 7 

mov ocx, odi 

call 7 ?0C5tr ingi$@QAC@XZ ; C3tring::C3tring(void) 
push offset Default 

muv ecx, udi 

mnv byte ptr [eepf 11!h4var_4] , H 

mov dword ptr [esi], offset off 404460 

mov dword ptr [esi+GCh], 0 

call V74i , ,xtringftWI < IAbAHVIh*HKIH4/j IhStri ng : :nperalnr~(.rhar rnnst. *) 
mov ocx, osi 

call sub_4035AO 

push 0 ; pvReserved 

call Colnilialize 

mov ecx, [esp+-10h+var_CJ 

mov eax, esi 

pop edi 

pnp esi 

mov largo fc:0, ocx 

add esp, 10 h 

rein a 
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As we enter sub_4018B0(), we see a fairly standard entry, and we note that this is employing 
the fastcall calling convention. After the procedure prologue, this also includes the setup of 
the SEH record. After this we see that the ECX register is saved and the parameter that was 
pushed onto the stack segment is copied into the EAX register. A copy of the ECX register is 
saved in the ESI register and the this pointer is saved in var_10 with Ynally then the constructor 
for the CDialog class being called. The two arguments passed to the constructor indicate that 
we donit want a particular parent window, or rather that the parent window should be that 
of the main application and the integer specifying a dialog template to use. Here weire just 
getting ready to setup a dialog box, which for being malware is a little odd to be honest. 
Weill keep looking through it though and eventually the pieces will come together. Moving 
on through the routine, we see an offset from the current ethisi pointer is calculated and 
saved in the ECX register, then var_14 is initialized to zero and the constructor for 
CString::CString(void) is called. If you were to single step into that constructor you would 
see Figure 8.16, which helps make sense ofwhatis going on in the binary. 


Figure 8.16 Single Step Constructor 

mFc42.dll:699FCB77 . - 

nfc42-dll:699FCB77 
nFc42.dll:699FCB77 mfc42 540: 


n>fc42.dll:699FCB79 push esi 

mFc42.dll:699FCB7fl now esi, ecx 

mfc42.dll:699FCB7C call near ptr unk_699FCftEF 

mFc42.dll:699FCB81 mou eax, esi 

mFc42.dll:699FCB83 pop esi 

mfc42.dll:699FCB84 retn 

mFc42.dll:699FCB84 ; - 

rafc42.dll:699FCB85 db 90h ; F 
nfc42.dll:699FCB86 db 90h ; E 
i»fc42.dll:699FCB87 db 90h ; f 

nFc42.dll:699FCB88 db 90h ; t 


I EAX 0012FE10 L»|stack[ OO00O740]: 0012FE10 

CF a 1 

. EBX FFFFFFOO 

PF 1 L 

ECX 0012FE70 l,|stack[ 00000740] : 0012FE70 

AF 0 

EDx|77030F34 l^jntdll .dll: ntdllJUFastSy 

ZF 11 

| ESI 0012FE10l»|stack[O0000740]:0012FE1O 

SF@ 

EDI 00406360 l»|uPX0:dword_406360 

TF 8 

EBP 0012FEFO |*jstack[ 00000740] :saued_Fp 

IF S 

ESP 0012FDDC U|stack[00000740]:0012FDDC 

DF 0 1 

EIP 699FCB77 U|nfcH2 .dll :nfc42_540 

1 Ifh nnnnn?4A 

OF Oj |! 


So knowing that in our context, the ECX register is an offset from the this pointer of 
the parent method. We can clearly see that a copy of it is made in the ESI register followed 
by a call to a near offset. However, interestingly enough, we see the apparent return value 
overwritten with the pointer that was passed into the method by the caller. Knowing that 
this is a constructor, we can almost look at this as being a placement new sort of thing. 
More likely however, is that our caller is a constructor in itself, which makes some sense as 
if you follow the CDialog: :CDialog() call in the calleris frame. You will Ynd a recursive call 
into sub_4018B0(). Another interesting aspect is that if you note, there are obviously several 
instance variables being instantiated in this method, to be more speciYc four instances of 
Cstring and three instances of CStringArray. However, the instance of CDialog is a little odd 
in that the ethisi pointer is not modiYd upon entry. This would at least imply that our class 
is some abstraction/interface/et cetera of the CDialog class. 

Throughout the routine, we see some oddities. For instance the this pointer is saved into 
var_10, but the variable is never referenced again, var_4 is incremented after each new object 
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is constructed, but never referenced again and so on. The point being that hopefully youire 
starting to get the picture about the different types of clues compilers leave laying around for 
you. Following this, we see the modiYcation of the data at the this pointeroit being modiYed 
to off_404460.What is interesting is that the pointer to the last CString object is overwritten 
with a NULL pointer, leaving an object with no reference. Although, we will reserve judgment 
until the destructor is inspected to make that determination. The call to the assignment operator 
for the CString object is also simple to Ygure out, if you were to single step in, youid see that 
quite simply this just ensures nil-termination for the string. Next, we see that the this pointer 
copy in the ESI register is copied into the ECX register and sub_4035A0() is called, implying 
it is another member method of this class. 

Stepping into sub_4035A0() we see the code in Figure 8.17, and it becomes obvious 
that what weire viewing is a string initialization routine, and furthermore that a somewhat 
vague attempt at making it obfuscated has been made, or at least that appears to be the case. 
This presumption is based off of the number of calls to the operator+= and all of the arguments 
being string constants. 


Figure 8.17 


HHur 

sub 403500 proc near 




push 

esi 




push 

edi 




nou 

edi, ecx 




call 

sub 40363O 




lea 

esi, [edi+60h] 




push 

offset dword 40626C 




nou 

ecx, esi 




call 

??VCString0@QOEOBU0@PBDBZ 

CString::operator+=(char 

const 

»> 

push 

offset dword 406264 




nou 

ecx, esi 




call 

YYYCStr1ngBBBOLOHUU0FBD0Z 

CString::operator*=(cnar 

const 

*) 

push 

offset dword 406260 




nou 

ecx, esi 




call 

??YCString@eqOEOBU0@PBD@Z 

CString::operator+=<char 

const 

*> 

push 

offset dword 406258 




nou 

ecx, esi 




call 

??YCString@@QOEOBU0@PBD@Z 

CString::operator*=<char 

const 

»> 

lea 

esi, [edi+64h] 




push 

offset dword 406254 




nou 

ecx, esi 




call 

??VCStringQ0QAEABUOBPBD0Z 

CString: :oper ator-*-=tchar 

const 

*> 

push 

offset dword 406250 




nou 

ecx, esi 




call 

??YCString@@QOEOBU0@PBD@Z 

CString: :operator*=(char 

const 

*> 

push 

offset dword 406244 




nou 

ecx, esi 




call 

??YCString@@QOEOBUOBPBDBZ 

CString::operator+=<char 

const 

*> 

push 

offset dword 40623C 




nou 

ecx, esi 




call 

??YCString@@QOEOBU0BPBD0Z 

CString::operator+=(char 

const: 

►j 

push 

offset dword 406238 




nou 

ecx, esi 




call 

YYYCStringBBQOEOBU0BPBDBZ 

CString::operator+=(char 

const 

»:• 

push 

offset dword 406230 




nou 

ecx, esi 




call 

??VCString@@QAEOBUOBPBDBZ 

CString::operator*=<char 

const 

*> 

pop 

eal 




pop 

esi 




retn 





sub_403500 endp 
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Upon entry into sub_3035A0(), we see a frameless method. That is to say that we do not 
see a strict procedure prologue or epilogue and we almost immediately call into sub_403630(), 
examining that yields a similar method as the one in Figure 8.18. 


Figure 8.18 

HI 


; CString::operator*=(char const ») 
; CString::operator*=(char const *) 
; CString::operator»-(char const *) 
; CString::operator*=(char const *) 
; CString;:operator+=(char const *) 

; CString::operator*=(char const *) 


sub_403630 proc neat- 

push esi 

push edi 

nou edi, ecx 

push offset dword_40628C 

lea esi, [edi+60h] 

nou ecx, esi 

call CString_equals 

push offset dword_406284 

nou ecx, esi 

call ??VCString@BQAEflBU0@PBD@Z 

push offset dword_406280 

nou ecx, esi 

call ??YCStringBBQAEABU0@PBD(az 

push offset dword_4062?C 

nou ecx, esi 

call ??YCStringB@QAEABlt0@PBDBZ 

push offset dword_406278 

nou ecx, esi 

call ??YCStringBeQAEABU0BPBDez 

push offset dword_406274 

nou ecx, esi 

call ??YCStringB8QAEABU06PBDBZ 

lea esi, [edi+64h] 

push offset dword_40628C 

nou ecx, esi 

call CString_equals 

push offset byte_4O6270 

nou ecx, esi 

call ??YCStringBeQflEflBU0GPBD@Z 

pop edi 

pop esi 

retn 

sub 403630 endp 


If you are really interested, you can single step through these two methods and analyze 
them, however all they do is result in two CStrings with the values ofihttp://www.alxup. 
com/adsnt/AdsNT.inii and ihttp://www.alxup.com/adsnt/AdsNT.exei respectively. 

Finally, upon returning back to the constructor method, we see a call to Coinitialize () 
and the this pointer is returned to the caller in the EAX register. Knowing what we know, 
weill call this method we just analyzed derrivedDialog::derrivedDialog(). Moving back to the 
calling routine, the WinMainQ function we have the code in Figure 8.19. 
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ecx, [esp+0E4h+def'riuedDialogPtr] 

derrioedDialog_derriuedDialog 

eax, [esp+OEOh+derriuedDialogPtr] 
ecx, [esp+0E0h+derriuedDialogPtr] 

[esp+0E0h+uar_4], 0 
[edi+20h], eax 

?OoModalQCDialog@@U0EHXZ ; CDialog::DoModal(void) 
ecx, [esp+0E0h+derriuedDialogPtr] 

[esp+0E0h+uar_4], OFFFFFFFFh 
sub 4019EO 


a n lu 

~1 

loc 40188E: 

mov 

ecx, [esp+0E0h+uar_C] 

pop 

edi 

pop 

esi 

xor 

eax, eax 

pop 

ebx 

mou 

large Fs:0, ecx 

add 

retn 

esp, 0D4h 

sub_401800 endp 


After the object construction, which is what we just analyzed we see that a copy of the 
pointer to the object is made in the EAX and ECX registers, that var_4 is assigned the value 
of zero and a copy of the this register is made directly prior to calling CDialog::DoModal(). 
We came into this situation hoping to get through the code learning as little about MFC as 
we can. However, this code presents us with a little bit of a problem, because weire almost at 
the end of the code and it looks like ids all going to be MFC-centric from here on. MSDN 
tells us that DoModalQ will create the dialog box that we just setup. So what we can expect 
to see are a series of call backs for handling the dialog. Leris Yre up the debugger and single 
step in again. 

After a lot of single-stepping, we Ynally get to a call in MFC that actually creates the 
window by calling CreateDialogIndirectParamA(). This API call sends a WM_INITI )I ALOG 
message to the dialogis appropriate procedure, thus this serves as our entry point into the 
GUI aspect of our application, which is the initialization callback for this dialog type. 
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Figure 8.20 Initialization Callback 


sub_401AFO: 
sub esp 
push esi 
push edi 


esp, 108h 

esi 

edi 

edi, ecx 

?0nInitDia1og@CDialog@@UAEHXZ; COialog: :0nlnit0ialog(void) 
7 AfxGetThread@@YGPAVCVinThread@@XZ; AfxGetThread(void) 
eax, eax 

short loc_401Bll 




1oc_401B13: 

mov eax, [eax+ZOh] 

push OFFFFFFECh 
push eax 


OFFFFFFECh nlndex 

eax ; hWnd 

GetWindouiLongA 

?AfxGetThread@@YGPAVCWinThread@@XZ; AfxGetThread(void) 
eax, eax 

short loc_401B31 



loc_401B33: 

mov eax, [eax+ZOh] 

push 80h ; 

dwNeuLong 

push 

OFFFFFFECh ; 

nlndex 

push 

eax ; 

hWnd 

call 

call 

SetWindowLongA 

7 AfxGetThread@@YGPAVCWinThreadSGXZ; Af xGetThreadCvoid) 

test 

jz 

eax, eax 

short loc_401B56 
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As we can see in Figure 8.20, the callback for initialization of the window is sub_401AF0(), 
which is another frameless method. Of what we see of the procedures initialization, there is not 
much that is interesting to us. We should single step through CDialog::OnInitDialog() to 
ensure that no derived procedures are called, and then also watch the indirect calls to EDX in 
the blocks labeled 0x00401B08, 0x00401B28 and 0x00401B4D. However, from what we can 
see thus far, itis mostly MFC code. 

Stepping through we Yid the following in Figure 8.21. The call to CDialog::OnInitDialog() 
did not execute any user-deYned code, and we can clearly see what the function pointer is 
in the following screenshot. It is important to note that because of the way that weire going 
through this code, and how fuzzy of a methodology weire employing, live set breakpoints on 
and around the entry points for all functions deYned in this binary. So at least in theory if 
we miss anything we will notice it and can back up to Ygure out what we missed, how we 
missed it and so on. 


Figure 8.21 


U 




H N Ld 1 

00401B08 

mou 

edx, [eax] 

00401B0A 

roov 

ecx, eax 

00401B0F 

jnp 

short loc 401B 1.3_ 


H N i.U 


00401B11 

00401B11 loc_401B11: 
00401B11 xor eax. 


dd offset ?GetHainMnd@cmnThread@@UflEPBUCUnd@@XZ ; Cl/inThread::GetHainWnd(uoid) 


Moving into the next section of code in this function, we Ynd the following code in 
Figure 8.22, which is a little more interesting and helps ETsh out the purpose more. 
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Figure 8.22 



Here we Ynd what we expected. Upon initialization of the window one of the Yrst things 
done is a call to MoveWindow(), which will resize and/or reposition a window (Figure 8.23). 
In this instance they resize the window to be zero by zero at position zero by zero. From there 
we obtain the location of the windows directory via a call to GetWindowsDirectoryA() and 
assign the return value to the CString at EDI+0x0A4, followed by a call to sub_401D00(). 
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Figure 8.23 



004G1D0G 
GG4G1DQ6 cnp 
GG4G1DQB setnb 
GG4G1D0E retn 
GG4G1DGE sub_401DQQ endp 
GG4G1DGE 


eax, 8GGGGGG0h 
al 


; and information about the operating system platform 


As we can see, sub_401D00() is a simple procedure that just checks to see if the current 
version ofWindows is at least Windows NT modifying the AL register based on this result 
and returning back to the caller. From there, we test the low-order 8-bits of the EAX register, 
either calling sub_401D10() or going to loc_401B9A() dependant on Windows version. 

If the version of the operating system is old enough, then all sub_401D10() does is call 
RegisterServiceProcessQ to keep the application from exiting when the user logs off. 
From there, we see that it opens the registry key iSoftwareXAlexa Toolbar! setting the variable 
at EDI+0x70 to 1 if the call succeeded and to zero if it did not. 


Figure 8.24 


61 N Ul 1 

QQ4G1BD7 






0G4G1BD7 

10C 401BD7: 


» 

time t * 

00401BD7 

push 

0 




004O1BO9 

call 

time 




00401BDF 

push 

eax 


» 

unsigned int 

00401BE 0 

call 

srand 




00401BE6 

nou 

ecx, [edi+20h] 



00401BE9 

add 

esp, 8 




00401BEC 

push 

0 



lpTimei-Func 

004O1BEE 

push 

28F20h 



uElapse 

00401BF3 

push 

1 



nIDEuent 

O0401BF5 

push 

ecx 



hWnd 

00401BF6 

call 

SetTimer 




00401BFC 

test 

esi, esi 




004O1BFE 

nou 

uIDEuent, 

eax 



004 QIC 03 

u 

short loc 

401C0C 




U 


[H N L4 

00401COS push 

esi 

; hKey 

00401C06 call 

RegCloseKey 



H N Ul 1 

00401COC 
00401C0C 

loc 

4O1C0C: 


OO4O1C0D 

pop 

edi 


00401 COD 

nou 

eax, 

i 

00401C12 

pop 

esi 


OO401C13 

add 

esp. 

108h 

0O401C19 

retn 


00401C19 

sub 

401AF 0 endp 

00401 Cl 9 



_| 
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From there, we see that the pseudo-random number generator (PRNG) is seeded with 
the current time and then the SetTimer() API is called. Here we note that a NULL pointer 
is passed in for the callback parameter, which means that a WM_TIMER message will be sent 
to the window when the timer expires in 0x2BF20 milliseconds or three minutes roughly. 
From there, the registry key is closed and the routine returns. From there, we will single step 
some more and see where we end up, knowing that in about three minutes tops we will 
land at the procedure for the WM_TIMER handler. 


Figure 8.25 


nf c42.dll:69C5C860 
► 4 nfc42.dll:69C5C860 

* nfc42.dll:69C5C862 
' nfc42.dll:69C5C864 

* nfc42.dll:69C5C866 
4 nfc42.dll:69C5C868 
‘ nfc42.dll:69C5C869 

* nfc42.dll:69C5C86F 
- 4 nf c42 .dll :69C5C871 

* nfc42.dll:69C5C873 
- nfc42.dll:69C5C876 

* nfc42.dll:69C5C878 

* nfc42.dll:69C5C87A 

* nfc42.dll:69C5C87C 

* nfc42.dll:69C5C881 

* nfc42.dll:69CSC883 

* nfc42.dll:69C5C888 
nfc42.dll:69C5C88f 
nfc42.dll:69C5C88F 
nfc42.dll:69C5C88F 

* nfc42.dll:69C5C893 

* nfc42.dll:69C5C895 

* nfc42.dll:69C5C898 

* nfc42.dll:69C5C89A 
4 nfc42.dll:69C5C89C 

* nfc42.dll:69C5C89E 
4 nfc42.dll:69C5C8A0 
4 nfc42.dll:69C5C8A3 
4 nfc42.dll:69C5C8A4 
4 nfc42.dll:69C5C8A5 
4 nfc42.dll:69C5C8AA 
4 nfc42.dll:69C5C8AB 

nfc42.dll:69C5C8B1 
nfc42.dll:69C5C8B1 
nfc42.dll:69C5C8B1 
► 4 nfc42.dll:69C5C8B1 
4 nfc42.dll:69C5C8B5 
4 nfc42.dll:69C5C8B7 
4 nfc42.dll:69C5C8B8 
4 nfc42.dll:69C5C8BA 
4 nfc42.dll:69C5C8BF 
4 nfc42.dll:69C5C8C1 
4 nfc42.dll:69C5C8C6 
4 nfc42.dll:69C5C8C9 
-* nfc42.dll:69C5C8CB 
nfc42.dll:69C5C8CD 
nfc42.dll:69C5C8CD 
nfc42.dll:69C5C8CD 
4 nfc42.dll:69C5C8D4 
nfc42.dll:69C5C8D6 
nfc42.dll:69C5C8D6 


loc 69CSC860: 

push 

0 

push 

0 

push 

0 

push 

0 

push 

ebx 

call 

ds:PeekMessage 

test 

eax, eax 

J nz 

short loc_69C5C8D6 

cnp 

[ebp-4], eax 

U 

short loc 69C5C88F 

push 

1 

nou 

ecx, esi 

call 

near ptr nfc42_6215 

nou 

ecx, esi 

call 

near ptr unk_69C5C206 

nou 

dword ptr [ebp-4], 0 

loc 69C5C88F: 

test 

byte ptr [ebp+8], 1 

jnz 

short loc_69C5C8B1 

nou 

eax, [ebp-OCh] 

test 

eax, eax 

jz 

short loc_69C5C8B1 

test 

edi, edi 

jnz 

short loc_69C5C8B1 

nou 

ecx, [esi+20h] 

push 

ecx 

push 

edi 

push 

121h 

push 

eax 

call 

ds :SendMessage 

loc_69C5C8B1: 

test 

byte ptr [ebp+8], 2 

jnz 

short loc_69C5C8CD 

push 

edi 

push 

0 

push 

36Ah 

nou 

ecx, esi 

call 

near ptr unk_69C3DA04 

add 

edi, 1 

test 

eax, eax 

jnz 

short loc_69C5C860 

loc 69C5C8CD: 

nou 

dword ptr [ebp-8], 0 

lea 

ebx, [ebx] 

loc 69C5C8D6: 


CODE XREF: nfc42.dll:69C5C8CBij 




; CODE XREF: nFcll2.dll:69C5C876tj 


CODE XREF: nFcM2.dll:69C5C893tj 
PlFc42 .dll: 69C5C89AT j ... 


CODE XREF: WFc42.dll:69C5C885tj 


CODE XREF: «lFc**2 .dll:69C5C85ETj 


nfc42.dll:69C5C8D6 




; mFc42.dll:69C5C871tj ... 

nfc42.dll:69C5C8DB 

nou 

edx. 

[eax] 


nfc42.dll:69C5C8DD 

nou 

ecx. 

eax 


nfc42.dll:69C5C8DF 

nou 

eax. 

[edx+64h] 


nfc42.dll:69C5C8E2 

call 

eax 



nfc42.dll:69C5C8E4 

test 

eax. 

eax 


nfc42.dll:69C5C8E6 

jz 

short 

loc_69C5C95B 


nfc42.dll:69C5C8E8 

cnp 

dword 

ptr [ebp-4], fl 
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As we single-step past the dialog creation, we enter a section of code in the MFC DLL that 
acts as a message queue loop, processing windows messages. After the call to PeekMessage(), if 
there is a message in the queue then control is branched off to loc_69C5C8D6, and ultimately 
the call EAX instruction calls CWinThread::PumpMessage(). Eventually, after walking through 
the loop long enough we Ynd the next callback, which is sub_401FC0() and is deYned as 
shown in Figure 8.26. 


Figure 8.26 


IB N i d 1 

O0401FC0 




00401FC0 




00401FC0 




O0401FC0 

; int 

Fastcall sub 40lFC0(char *) 

O0401FC0 

sub 401FC0 proc near 


00401FC0 




HH4 01m h 

uar 4= 

flwora ptr -4 


00401FC0 




00401TC1 

push 

esi 


O0H01FC2 

roov 

esi, ecx 


004 01FC4 

push 

edi 


00401FC5 

push 

ecx ; 

char * 

O0401Fr.fi 

1 pa 

prli , [psi + 0A4h] 


00401FCC 

ID0U 

eax. esp 


00401FCE 

mou 

Tesp+IOh+uar 41, 

esp 

00401FD2 

push 

offset alndex htn 

; "Windex.htn" 

00401FD7 

push 

edi 


00401FD8 

push 

eax 


00401FD9 

call 

??H@YG?AUCString@@ABUO@PBD@Z ; operator+(CString const &,char const *) 

00401f Dt 

lea 

ecx, [esi+oonj 


Q04O1FE1 

call 

sub 401F90 


00401FE6 

push 

eax ; 

lpszdrl 

00401TC7 

mou 

ecx, esi 


00401FE9 

coll 

sub 402850 


00401FEE 

test 

eax, eax 


004 01 F F 0 

jnz 

short loc_40201D 



t 


JL 


IB N id 1 

0O401FF2 

push 

offset alndex htn ; "\Sindex. 

.htm" 

00401FF7 

lea 

ecx, [esp*10h*var 4] 


00401FFR 

push 

prti 


004O1FFC 

push 

ecx 


0O401FFD 

call 

??H@VG?AUCString@GflBU0GPBD@Z 

: operator*(CString const &.char const *) 

00402002 

nou 

edx, reaxl 


00402004 

push 

edx ; lpFileNane 


00402005 

call 

DeleteFileO 


0O402O0B 

lea 

ecx, [esp+OCh+uar 4] 


0O4U2 00f 

call 

YYiustringi^gotyxz ; cstring: 

:”i;iitring(uoid> 

00402 014 

nou 

eax, i 


00402019 

pup 

edi 


00402010 

pop 

esi 


00402 01D 

pop 

ecx 


004 02 01C 

retn 




HNul T _ 

00402 01 D 

0040201D loc_4 02O1D: 
OO40201D pnp prti 

0040201E xor eax. eax 

O04O2020 pop esi 

00402021 pop ecx 

00402022 retn 
00402022 sub_401FC0 endp 
00402022 


Upon entry into the function, we see a bit more that interests us. SpeciYcally, one of the 
Yrst things done is the string i\index.html and the Windows directory are concatenated 
together. This yields the string iC:\Windows\index.htmi, which is followed by a call to the 
sub_401F90().This simply dereferences the pointer in the ECX register and copies that 
pointer into the EAX register. Then a pointer to the string http://www.alxup.com/adsnt/ 
AdsNT.ini and the newly constructed string are then passed as parameters to the routine 
sub_402B50() which we examine in Figure 8.27. 
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HNul 

OOJ*02B5O 

00462850 

00402850 

00402850 ; int _stdcall sub 402B50(LPCSTR IpszUrl.char «) 

00402B50 sub_4O2B50 proc near 
00402850 

fln407R50 i1uNiirahprnfRytPsRpart= riwnrri ptr 41Ch 

00402850 Buffer- dword ptr 418h 

00402850 uar_4l4= dword ptr -414h 

00402B50 mmufferLengtn*= dword ptr *4ion 

00402050 var_40O dword ptr 40Ch 

00402850 oar_H= riwnrri ptr -OHh 

00402B50 var_4= dwurd plr -4 

00402850 lps 2 Url= dword ptr 4 

00402850 arg_4= dword ptr 8 

00402050 


00402852 

00402857 

00402850 

0O402O5C 

00402B65 

00402B6B 

00402B6C 

00402860 

0O40286E 

00402B70 

00402872 

00402874 

00402876 

Q0402B7B 

00402880 

00402886 

00402888 

AR402RRA 

00402B8E 


uffsel lot 403E1 
eax, large fs:0 
eax 

large fs:0, esp 
esp, 41 Oh 
ebx 
ebp 
edi 
0 


offset szflgent : adsn 
ebp t 1 

InternetOpenO 

ebx. eax 
ebx, ebx 

[psp*42Rh+uar_4i4], phx 
loc 402C62 


dwFlags 

lpszProxyBypass 

IpszProxy 

di/AccessTypc 

"adsntB/1.6" 


w- 

GJNU1 

00402B94 

nou 

eax. 

[esp*428h*lpszurl] 

00402B9B 

push 

0 

; dwContext 

00402RQD 

push 

R4nnniRRh ; dwFlags 

O0402BA2 push 

0 

; dwHeadersLength 

004O2BA4 

push 

0 

; lpszHeaders 

00402886 

push 

eax 

; lpszllrl 

004 02007 

push 

ebx 

; hlnternet 

flMlR?R0R 

rail 

Internet npenllrl 0 

0O4O2BOE 

led 

eux. 

[esp+428li*dw8uf ferLeiiylh] 

O04O2BB2 

push 

0 

; lpdwlndex 

00402884 

lea 

edx. 

iesp*42lTH8iifforj 

00402080 

push 

ecx 

; lpdwDufTerLength 

(10Un2RR9 

nnu 

pdi, 

eax 

0O402B6B 

push 

edx 

; lpBuffer 

00402BBC 

push 


bClSu ; dwInfoLeuel 

0U402BU1 

push 

edi 

: hRequest 

00402DC2 

nou 

i es p 

•43Ch>dwDuiferLcngthl, 20h 

0QU02BCA 

call 

HttpQuerylnfoA 

0O402BD0 

inp 

fesp*-42Hh*Buf Ferl, 190li 

O04O2BD8 

jnb 

loc_ 

402C62 

-1—I 



As we step into this new routine, it becomes obvious that weire really starting to get 
down to brass tacks with the application. We can now see it making contact with the outside 
world. After the procedure prologue we see a call to InternetOpenA().The only interesting 
parameter to us in this instance is the User-Agent that will be employed. This gives us a 
static network detectable pattern to look for. For instance, suppose you wanted to generate 
an IDS rule, you would probably look for a packet that looks something like this (nuances 
of fragmentation/request splitting/weirdo tab placement/urgent data/et cetera ignored): 

GET /adsnt/AdsNT . ini [... ] 

[...] 

User-Agent: adsntB/1.6 
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The InternetOpenA() API call returns a handle, and we can see the return value copied 
into the EBX register. This is eventually copied into var_414, and renamed to einetHandlei. 

It should also be noted that the return value is checked against NULL. From there, the URL 
that was passed in as a parameter is retrieved via a call to InternetOpenUrlAQ. After that has 
been retrieved, a call to HttpQuerylnfoAQ is made checking the status code returned by the 
remote web server. The code then checks this value against 0x190, which is 400 decimal and 
branches if the status was not less than 400. Now potentially (and Km not positive because 
I would need to delve further into the InternetOpenURLQ function than I am currently 
willing to), if the status code returned was larger than 0x20 in length than we would end up 
with an uninitialized variable, which could result in a bug. This is unlikely in this case 
because the Ehgs passed to HttpQuerylnfoAQ speciYes that we want the status code returned 
as an integer (0x20000013 is HTTP_QUERY_FLAG_NUMBER | HTTP_QUERY_ 
STATUS_CODE). Moving on through the function body, we see Figure 8.28. 

Figure 8.28 


i a n fi 


BOWODC test 
imnn?BFn j7 


edi, edi 
short lor 402058 


mi _ 1 

004O2BE9 

push 

esl 

004Q2BE0 

push 

oFFsct aWb , "wh" 

004O2BLF 

push 

eax : cnar « 

0O4O2BF 0 

nov 

[esp*434h*dwMiunbe» 0fBytesRead], ebp 

004O2BF4 

call 

fopen 

00402BFR 

nou 

esl, eax 

004O2BFO 

add 

esp, 8 

004U2BFF 

test 

esi. esi 

00402001 

jz 

short loc 402053 



004020 03 ituu 

eax, 

[ ebp*42Ch*dwHui'rt>er OFBy LesReaU ] 1 

00402007 test 

eax, 

eax 


ou402uuy joe 

short 

100 4U2C47 





L l_ 

_I_ 



IH N 1.4 



004020OR 

mnu 

ehx, InlernPlRpadFilp 



00402011 

nou 

ebp, Fw» it? 



hnul 


3 


00402C 17 
00402C17 
OH402C1/ 
0040201D 
00402C1F 
00402020 
O0402C25 
UU402026 
00402C27 
0 04 02 C 2 9 
00402021) 
O0402C2E 
U0402U2F 
00402030 
0040202'; 
00402C36 
00402038 
UU4U2U3U 
O0402C3r 
00402041 


loc 402C17: 


lea 

lea 

push 

push 

push 

push 

call 

mou 

push 

push 

lea 

push 

push 

call 

now 

add 

test 

U 


PCX , 

edx, 

ecx 

3PFIi 

edx 

edi 

ebx 

eax, 

esl 

eax 

ecx, 

1 

PCX 

ebp 

eax, 

esp. 

eax, 

shnrl 


Lesp>42Cn*duHunDerUlBytesKeadj 
[esp•42Ch'inetOuf f er] 

; IpduNunberOFBytesRead 
, UwNuwberOfBylesTuKead 
; lpBuffcr 
.* hlile 

InternetReadflie 
[esp*42Ch*duNunberOFBytesRead] 

; FILE * 

; size_t 

[esp*434n*inetUuFFerj 
; size_t 
: unid * 

fwriLe 

[esp«-43Ch*duNunborUfBytesRead] 

ion 

eax 

Inn 402017 


— 
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Moving down, the next thing we see is that the copy of the handle returned by 
InternetOpenA () is tested, branching if its NULL and otherwise moving on to a call to fopenQ. 
Here we see the second parameter come into play as a Me with that name is opened for binary 
writing. Next the number of bytes is checked, and if all is well we end up at loc_402C17. 
Here we see a simple loop where 0x3FF bytes at a time are read from the network and, 
then written to disk via calls to InternetReadFile() and fwriteQ respectively From this loop 
we enter the next section of code which is shown in Figure 8.29. 

Figure 8.29 



Moving into the Ynal portion of the function, we see largely what we expected. Having 
read all of the Me from the network and writing it to disk, we now close the Me handle and 
internet connection, then call the destructor for the CString and return the EBP register as 
the return value. This is zeroed out at 0x00402C51, thus a successful I/O cycle in this routine 
returns a value of zero, and non-zero for failure. So in conclusion, this function takes two 
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parameters: a URL and a path. The routine retrieves the Me at the URL and saves it to the 
path speciYed.We will rename the function downloadToFile(). It should probably be noted 
that at least under Vista, there is some oddity that causes the Me to be written to iC:\Users\ 
[username]\AppData\Local\VirtualStore\i instead of C:\Windows. That all said, letis take a 
look at the INI Me we downloaded. 

[AdsNT] 

Version=l00 
AdNum=9 
[AdsNTURL] 
leftposl=-1000 
topposl=-1000 
widthl=l 
heightl=l 

urll=http : //www.deepdo.com/union/3721/yad.htm 

objurll=http://zzz.yy.xom 

weightl=l0 

showIEWindowl=0 

showStylel=0 

groupl=0 

leftpos2=-1000 

toppos2=-1000 

width2=l 

height2=l 

url2=http://talent.deepdo.com 

objurl2=http ://xx. you.com 

weight2=l00 

showIEWindow2=0 

showStyle2=0 

group2=0 

leftpos3=-1000 

toppos3=-1000 

width3=l 

height3=l 

url3=http : //www.deepdo.com/union/3721/yad.htm 

objurl3=http://xxx.yyy.com/ 

weight3=l0 

showIEWindow3=0 

showStyle3=0 

group3=0 

leftpos4=-1000 
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toppos4=-1000 

width4=l 

height4=l 

url4=http://www.92site.cn/search/135go.jsp 

objurl4=http://xxx.com 

weight4=15 

showIEWindow4=0 

showStyle4=0 

group4=0 

leftpos5=-1000 

toppos5=-1000 

width5=l 

height5=l 

url5=http://www.5isou.cn/ 

objurl5=http://xxx.xxx.com/ 

weight5=l00 

showIEWindow5=0 

showStyle5=0 

group5=0 

leftpos6=-1000 

toppos6=-1000 

width6=1 

height6=l 

url6=http://www.deepdo.com/site.htm 

objurl6=http://xxx.xxx.com/ 

weight6=150 

showIEWindow6=0 

showStyle6=0 

group6=0 

leftpos7=-1000 

toppos7=-1000 

width7=l 

height7=l 

url7=http://www .5isou.cn/calendar/index.htm 

objurl7=http://xxx.xxx.com/ 

weight7=150 

showIEWindow7=0 

showStyle7=0 
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group7=0 

leftpos8=-1000 

toppos8=-1000 

width8=l 

height8=l 

url8=http://www.92site.cn/search/135go.jsp 

objurl8=http://xxx.xxx.com/ 

weight8=150 

showIEWindow8=0 

showStyle8=0 

group8=0 

leftpos9=-1000 

toppos9=-1000 

width9=l 

height9=l 

url9=http : //www.iesafe.cn/yahoo/index.htm 

objurl9=http://xxx.xxx.com/ 

weight9=150 

showIEWindow9=0 

showStyle9=0 

group9=0 

[AdsNTGroupURL] 

gurl2l=http://www.deepdo.com/union/iplus/edodo.htm 

As you can see, in our Yle we have a list of URLs and a series of positions and sizes; 
we can probably safely assume then that weire looking at ad-ware, and that the sizes/positions 
refer to browser window placement. Whatis annoying, though, is that the application weire 
going to download, probably downloads some other application, which downloads another 
and so on. Even more, we can bet that many of these URLs also host more malicious content. 
Itis really a losing battle for the user who made the mistake of clicking something they didnit 
mean to once as their computer ends up overloaded. This is interesting because if you look 
at many viruses and worms, the idea of coexistence with the host platform is something that 
can often be found, to the point that there are actually viruses that attempt to remove other 
viruses that they recognize. Adware and Spyware are the exact opposite. I suppose it could be 
argued that the motives are different and that greed causes the corporate malware to take the 
approach of infecting the computer to the point of it being useless as they donit expect to 
survive for any period of time anyways. Nonetheless, enough with the theory, letis get back 
to the application shall we? 

Moving back to the calling function we have the code in Figure 8.30 left to analyze. 
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0O401IT2 push 
00401FF7 lea 
0O401ITD push 
00401FFC push 
0O40irro can 
00402 002 PIOU 
00402004 push 
00402005 call 
00402000 lea 
00402OOF call 
00402 014 mow 
00402019 111)11 
00402010 pop 
0040201R 111>11 
0O40201C Ketn 


edx, [eax] 

edx ; IprileNane 

OeleteFllea 

ecx. [esp«OCh*var_4] 

??icstrlngaaQ0EQXZ curing-cstrlng(uold) 

eax, 1 

Hilt 

csi 


So, after the return value is checked and the call to downloadToFile() succeeded, we 
continue execution at 0x00401FF2, or loc_40201D if the call failed. The CString for the 
path is once again constructed and a call to DeleteFileA() is called, removing the Me. After 
that the destructor for the CString is called and we return to the caller, which the message 
handling code for the application. From there we enter the next function, sub_401DF0(), 
which is deYned as shown in Figure 8.31. 

Figure 8.31 


.!' • 


004M1DI II 

OO401DF 0 

004O1DF 0 

0O4O1DFO sub 401DF0 prut near 

U04O1OF 0 

00401DF0 IpExistinqFileNane* duord ptr -ICh 

00401DI N uar_1H- rtwnrd ptr lllh 
mi4inrjFit ipFIlcHonc- nwnrrt ptr -14h 

00401DF0 war 10- dvord ptr 10H 

1)0401 OF 0 war O dvuril plr - OCh 

0O401DFO var 4- dvurd ptr -4 

00401DF 0 


Hint midi v push 

ollset lor 4lCinilli 

00401OF7 nov 

cax. large fs:o 

004010FD push 

cax 

00401DFE now 

large fs;0, esp 

00401 EOS sub 

esp. lOh 

00401E08 push 

ebx 

004011 IN push 

esi 

0 0401 r lift now 

esl, erx 

OD4U1EOC push 

edl 

0040IE OD push 

uffset aAdsul ini . ’WAUtlll .ini’ 

004U1E12 lea 

eax. (esp*2Ch*lpF1 leMane) 

00401E16 lea 

edl, (esi♦ Oil44] 

UU4U1L1C push 

edi 

004011 11) push 

eax 

00401E1E call 

??HOYG?flUCStrin<)(?WiBVO0PBDQZ ; operator*<Gitrlnq rnir ' h.'iiai > oust • i 

00401E23 push 

ecx ; dcstFlle 

00401E24 lea 

edx. [esp*2Cli«lpFileNane] 

00401E28 now 

ecx, esp 

00401E2A mu 

[esp*2Ch*uar 10], esp 

004011 VI push 

edx 

004011 21 nou 

[esp*:inr.»uar 4], n 

00401E37 call 

??ocstringe0onEWiBVO002 , CStrinq::citrlng(GbtiTnq const m 

00401E3C lea 

ecx, [usi-AWi] 

00401E3F nov 

byte plr [esp*2Ch*var 4], u 

U04U1E44 call 

sub 401F90 

ou4uit4y push 

eax ; lpszun 

11114 111 1 40 mu 

erx, psi 

00401E4C call 

dovnloadloFile 

00401ES1 tCOt 

eax. eax 

004 0IES3 jiu 

short loc_401E78 


U0401 ESS MV 
00401ES9 push 
O04U1E5A push 
00401ESC push 
UU4V1L61 push 
HM4II1I 6A ran 
00401E6C cnp 
|U0401E72 jgc 


cox. [csp«2Hh*ipFileNane) 
vox ; lpFileHane 

0 ; nDefault 

offset KepNane M U*rsion" 
oil set flppNane ; "AUsMI" 
Krrrrluarerrol 1 1 »I nrn 
dvord 40608C, cox 
1OCJ401F6C 
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Upon entering into this new subroutine, we Yrst see that the path to the AdsNT.ini Me 
is again constructed and then downloaded, branching off to loc_401E78 if there was an error, 
otherwise continuing at 0x00401E55. From there, we see that the Version is retrieved from 
the INI Me via a call to GetPrivateProMelntAQ and then a conditional branch based off of 
that, thus we have a simple version compatibility check. Moving on to the next section, we 
see the code from Figure 8.32. 

Figure 8.32 


call 

push 

lea 

mov 

call 

push 

push 

call 

push 


push 


loc_401E78 

lei ©ok, [esp+23lHvar_18] 

CString__CString 
lU4h 

ecx, Lesp+ZCrH-var.lSj 
byte ptr [esp-+Z'JOvar_4 ], 2 

?G©xBui-fer'icsxr1ng<J«CrtEPftDWZ; CString: :GetBuifer(1m) 
eax ; ipBuffer 

I04h ; neuf-ferLength 

GelTeinpPallri 
OFFFF FFFFh 

ecx, [ eep*2C I r*-vai _ 18 J 

*?Rel6aeeDurfer(3CGT.n'ngi3«QAD<l*«Z:; COiMng ; Rel&as«Du I <@-(int) 
vex, [eept28lM-lpEx te» iiyF ilwNmim] 

CQtr1ng_Cutring 

ecKj [e©pi2Shivar_18] 
of f©e t a4dsn i_sueJ "\\AdsNT . one * 
edH, [e©p'2Ch'var_10] 
bl» S 

OCX 

odh 

byt© pir [oep+34h«-var_4], bl 

R^flVr.5rr.rinr](W^Vn(!f'RnflP; nporatnr+<r^tt.r ing rmtl fi ,r.har rnnct ") 

oax 

oth . [ocp+9Tf>+lpFxislingFilaNamo] 
byte ptr [esp+2Ch+var 4], 4 

9 , 74nf?t.r in«](WI>aFARVf«4RVi1ftfl?; CString • ■npoMinrr(f!ftT.rirG cnnct #) 
ecK. [esp*23h+var 10] 
byte ptr r&Bp+28h+var_4l, bl 
T?lCStrinq<}SWE®XZ : CStrinq: :''C3trinq< void) 
ecH ; destFile 

eax, [esp+2Ch+lpExi8tirigFilafoatne] 
sex, esp 

[esp*2Chivar_10], eep 

■??OCString»J<5CWE®fleV0<JdZ; CString: :C8tring(C3tring cone*. R) 
ec«, [esi+64h] 

byte ptr |.esp+2Chtvar_4J, bl 
6UD_4Ul»-aU 

eax j IpszUrl 

ecx, esi 
doxnioadToM 19 

©ax , wax 

snort ioc_40iF46 


MOV 

P'jch 

puch 

MOV 

rail 

puch 


MOV 

nail 

lea 

MO 

call 

push 

lea 

MO < 
MO < 

push 

nil 

lea 

MO J 

cal l 
push 

mO l 

call 


Jnz 


*a Isa 


OM01F13: 

push offset aAdsnt.sxs; "WfldsNT .exe ‘ 
ecx, [esp-»2Cb+var_10J 
edl 
ecx 

r(W(stijfAVUJtring«<S4bvu(.*r'BU«d; operator+(i»tring const K.cnar const 1 
eax, [eax] 

sax, [esp+ 2 t-n+ iptxistingf- i leWame j 
o : int 

eax char •* 

edx ipExIst IngFlieUame 

byte p>.i [eepi-341'i rdi _4] 6 

sub_4 01010 

ecx, [eept-23lH-vai _ 10] 
oyte p»r [esp-t-ZOhtvar.-*], bl 
'??lC3tring | 3$QAE<5X2 ; COtringi !"DString< void) 
abort loc_401FS0 


push 
push 
ca 1 1 

MIC V 

mov 

push 

push 

push 

iiiuv 

call 

lea 

mov 

call 


loe_401F46' 

Imov const anf_l, 


1OC.401F6C; 

lea ecx, [esp+28h*-lpFil9Nams] 

hov [espi28h+var _4J, UFFFFFFFFh 

call ' 1 ‘?lCString | 3<SQAE<XZ, CString :"CString(void) 

Bov ecx, lespt2S|-rivar_C] 

pop odi 

pop esi 

bov large fs:U, eex 

pop aox 

add esp, icn 


retn 


loc_4Uii-oU: 

lea ecx, [esp+ 2 «n+iptxtstirtgri lerwine] 

nov ovte ptr [esp+28h+var_4] z 

L.all T!1C3U hiy€«QAE«XZ: C3t. iuy. "C3U-i.iy< vu Id) 

lea ecx, [eepi2Sn-*var_i8] 

liuv byte pir [eep+28liiva( _4] , 0 

oall r!lC3ving4<5Q«K5XZ:; COtr-ing; "C3tring<void) 
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Starting at loc_401E78, we see a fairly familiar pattern, we see a series of CStringB being 
created, although we see a slight twist in that a temporary Mename is retrieved via a call to 
GetTempPathAQ. From this path we ultimately note that a temporary Mename and the 
string i\AdsNT.exe! are concatenated together. From there, we can see exactly what is going 
on by the call to downloadToFile(), which if you remember was the routine we reversed 
immediately prior to this routine and that simply acted as a go between for a URL and a 
local Me on the disk, fetching a remote Me and saving it. Thus the actual executable image 
that came up at the Yrst object creation is Mnally being downloaded. 

Thus far we have concluded that the application is deMnitely malicious in the sense that 
it is obviously adware. It was packed and attempted to obfuscate the strings stored internally, the 
later especially should raise red Ehgs for you. Furthermore, an INI Me was downloaded that 
after review it became quite clear that it contained information such as URLs and coordinate 
and window sizes presumably for those URLs. Things look pretty suspicious, but there is still 
a lot of work to be done. This is okay though, as it provides an excellent opportunity for you, 
the reader, to apply the knowledge learned in this book. On the Syngress website you can 
Mnd a copy of this program and repeat some of the steps we have, such as unpacking the 
application. From there, consider the following exercises: 

Exercise 0: Starting at 0x00401F13, reverse the rest of the application-how does the 
rest of it operate? 

Exercise 1: Reverse/Analyze AdsNT.exe 

Exercise 2: In dealing with the message queue, we skipped over a section of code 
that is actually deMned by the application and serves as the callback that ultimately 
calls the majority of the functions weive analyzed here. Starting at the end of the 
Yrst routine to call downloadToFile(), the one that Mrst obtained the INI Me, walk 
through until the routine returns and make note of where in the application control 
Ebw is handed to, not step backwards through the application until the initial entry 
point is realized. 
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Introduction 

IDA Pro is a tool used by many people in different areas of reverse engineering. The user base 
includes malware analysists, vulnerability researches, software reversers, hackers, firmware/ 
hardware reversers, developers, software protection enthusiasts, and many more. 

IDA Pro’s extensibility is what makes it a great tool. The interactive nature of IDA goes very 
well with scripting and writing plug-ins. IDA is a tool in the true sense of the word. The user 
guides IDA to achieve what is needed. 

This chapter is about extending IDA Pro with scripts or plug-ins. As we reverse engineer 
binaries, eventually we start seeing patterns. These patterns can be code patterns or repetitive 
tasks that are ripe for automation. 

IDA can be extended using various methods. IDC is the built in scripting language. IDC 
is C-like in structure and since it is interpreted, no other tools are needed. More complicated 
tasks are relegated to plug-ins. Hex-rays provides an SDK to customers allowing for plug-in 
development. The SDK is written in C++ with support for various compilers.Third party 
hybrid solutions have also been developed. These hybrid solutions wrap IDC functions as well 
as some SDK functions. (You can download code and scripts in this chapter from 
www.syngress.com/ solutions). 

Basics of IDA Scripting 

IDC is IDA Pro’s built in scripting language. It is very similar to C in syntax. Someone familiar 
with C should be able to pick up IDC quickly. It is interpreted. 

There are two standard ways to execute IDC. 

■ IDC Statements can be executed direcdy from within IDA. SHIFT+F2 brings up a 
dialog box. Statements entered in the box will be executed. The dialog box is used to 
enter in small code snippets. Functions are generally not defined in the dialog box, 
although there is a w 

■ IDC files can be loaded. To load an IDC file go to File | IDC File .A file browse 
dialog will come up. 


Another option to execute IDC expressions is through an optional command 
line. The command line option must be set to yes in idagui.cfg. 

DISPLAY_COMMAND_LINE = YES// Display the expressions/I DC command line. 
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I DC Syntax 

The IDC scripting language borrows a great deal of syntax from C. All statements end with 
a semicolon. The similarity includes many of the same keywords including if, if else, while, 
do while, continue and break. This section will introduce IDC syntax highlighting differences 
between IDC and C. 

Scripting provides access to the disassembly with much less effort than writing plug-ins. 
Scripts can be run from files as well as the IDC dialog box. The examples in this section will use 
the dialog box until we reach functions. The use of even simple scripting will speed up analysis 
and help automate tasks. In order to run IDC scripts, an idb file must be loaded into IDA. 

Output 

The first thing taught in most programming books, since K&R C, is the hello world 
program. Getting data out to the user from a script is very important. This can be actual 
output or even just for debugging purposes. 

Open the IDC command window by pressing SHIFT+F2 or using the menus (File | IDC 
Command...) and type: 

Message("Hello world\n"); 

The dialog should appear similar to Figure 9.1. Multiple statements can be entered, but 
for now just the Message statement will suffice. After clicking OK, hello world should appear 
in the message window. 

Figure 9.1 Hello world 


Please enter text _ □ X 


Please enter IDC statement(s) 

Message("hello world\n"); 


CK 


Cancel 


Help 
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The Message function is similar to the C printf function, also using format strings. 
The function prototype is: 
void Message (string format,...); 

Some other variants using Message: 

Message("%s\n", "hello world"); 

Message("%x\n", 0x40100); 

Message ("%x is the cursorVs address\n", ScreenEAO); 

The first example uses the “%s” format string also printing hello uwW.The second example 
uses “%x” to print out a hexadecimal value. While the other two examples are somewhat 
contrived, the third one uses a new IDC function. The ScreenEA function returns the current 
cursor address. This function is commonly found in scripts. 

Message is not the only output, but it is the most commonly used. Two other functions 
are available, Warning and Fatal. Both of these use format strings and have the same function 
prototype. 

Warning is used to alert the user of problems. It will bring up a box similar to Figure 9.2. 
Fatal is rarely used since it terminates IDA without saving the database. 


Figure 9.2 Warning Box 


Warning 


m 


This is a warning 

l~~) Don't display this message again 


OK 


Variables 

All variables in IDC are defined using the auto type. The statement below declares a variable 
called counter. 

auto counter; 


The auto type can represent 

Example 

32bit integer (64 bit in IDA Pro 64) 

0x00401000 

character string 

"hello world" 

floating point number 

5.23 
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Variables have size limits depending on the type of data they contain. 

■ Integers are 32 bit (64 bit for IDA Pro 64). 

■ Character strings can be up to 1023 characters long. 

■ Floating point variables are up to 25 decimal digits. 

An auto variable can represent different types of data. As such, there are conversion rules 
when operating on different types. Generally when scripting, type conversions are not as 
common as in C. There are functions to manually perform type conversion: 

long(expr) 
char(expr) 
float (expr) 

Variables must be declared and assigned in separate statements. 

auto currentAddress; 
currentAddress=ScreenEA(); 

Most of the standard C operators (+, —, /, *, %, «, », ++, —) work in IDC. Some 
operators are unsupported, these include the combination assignment operators (+=) and 
comma operation (,). Unlike C, added strings will be concatenated. 

All variables have local scope. This means they are only available within the function that 
defined them. For our current purposes, this applies to the IDC command window. Functions 
are covered later as well as way to allow global variables. 

Conditionals 

Most of the standard C conditional statements are available. These include if if else, and the 
ternary operator “? :”.The switch statement is not available in C.This code snippet shows 
if else. 

auto currAddr; 
currAddr = ScreenEA(); 
if (currAddr % 2) 

Message("%x is odd\n", currAddr); 
else 

Message("%x is evenin'', currAddr); 

Loops 

Looping can be done by for, while, and do while. These are similar to C except the comma 
operator is not allowed. The switch statement is not supported in IDC, but multiple if, else 
if statements can be used. The code snippet demonstrates a loop and introduces some new 
IDC functions and concepts. 


www.syngress.com 


204 Chapter 9 • IDA Scripting and Plug-ins 


auto origEA, currEA, funcStart, funcEnd; 
origEA = ScreenEA(); 

funcStart = GetFunctionAttr(origEA, FUNCATTR_START); 
funcEnd = GetFunctionAttr(origEA, FUNCATTR_END); 

if (funcStart == -1) 

Message("%x is not part of a function\n", origEA); 
for(currEA=funcStart; currEA != BADADDR;currEA=NextHead(currEA, funcEnd)) 
{ 

Message("%8x\n", currEA); 

} 


Note 

BADADDR is a constant used in IDC. It represents an error or invalid result from 
function. In scripts it is used to test results and sometimes initially assigned to 
variables. 

Some IDC functions return -1 on an error. Internally BADADDR is represented 
by -1. 


The code snippet prints out the addresses to every instruction within the current function. 
It introduces two new IDC functions, GetFunctionAttr and NextHead. The prototypes are: 

long GetFunctionAttr(long ea, long attr); 
long NextHead(long ea, long maxea); 

GetFunctionAttr allows us to query for certain function attributes. The argument ea is any 
address within the function. The argument attr is the specific attribute we are interested in. 

In this case we are looking for a function start and end given an address. If the address ea is 
not within a function then GetFunctionAttr returns —1. 

NextHead returns the next instruction or data item. The argument ea is the start address 
and maxea is the end address. If there are no defined instructions or data in the given address 
range, then BADADDR is returned. On architectures like the IA-32 containing variable 
length instructions NextHead must be used to iterate through instructions. On RISC archi¬ 
tectures with set length instructions one may be tempted to simply increment instead of 
using NextHead. This should be avoided as simply incrementing will not check if the item 
has been defined by IDA. 

The code snippet demonstrates the same functionality using a while loop. 

auto origEA, currEA, funcStart, funcEnd; 
origEA = ScreenEA(); 
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funcStart = GetFunctionAttr(origEA, FUNCATTR_START); 
funcEnd = GetFunctionAttr(origEA, FUNCATTR_END); 

if (funcStart == -1) 

Message("%x is not part of a function\n", origEA); 
currEA = funcStart; 
while (currEA != BADADDR) 


Message("%8x\n", currEA); 

currEA = NextHead(currEA, funcEnd); 

} 

Functions 

Functions are needed once we move on from simple snippets of IDC. Functions are also 
required within IDC files. All functions in IDC are defined as static. The code below is an 
example function. 

static outputCurrentAddress(myString) 


auto currAddress; 
currAddress = ScreenEA(); 

Message("%x %s\n", currAddress, myString); 
return currAddress; 

} 

Function declarations in IDC have a few differences with C.The differences relate to types. 
Since IDC has only one type, auto, types are not needed in the arguments or return. IDA only 
accepts functions without types in the declaration. IDC files will be covered in the “Simple 
Script Examples” section later in this chapter. 

Usually functions are only declared in an IDC file. Entering the above function in the IDC 
Command window produces an error “Syntax error near static”. This is due to how the 
IDC Command window operates. A solution was first documented by Willem Jan Hengeveld 
(http://www.xs4all.nl/~itsme/projects/disassemblers/ida.html). 

Internally the dialog box contents are stored in a function called _idc. Thus entering in 
a function declaration is actually attempting to declare a function within another function. 
The _idc function needs to be closed before a new function is declared. The new function 
must also leave off its closing brace. In order to declare outputCurrentAddress enter: 

} 

static outputCurrentAddress(myString) 


auto currAddress; 
currAddress = ScreenEA(); 
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Message("%x %s\n", currAddress, myString); 
return currAddress; 

We should not see an error. Although we declare the function it does not execute. If we 
want to execute something as well, it needs to be part of the _idc function. 

AddHotkey("Alt-f9", "outputCurrentAddress2"); 
outputCurrentAddress2(); 

} 

static outputCurrentAddress2() 


auto currAddress; 
currAddress = ScreenEAO; 

Message("%x\n", currAddress); 
return currAddress; 

The preceding code introduces a new IDC function, AddHotKey .This function binds a key 
combination to an IDC function name. The target function can not have arguments. The hotkey 
is added and outputCurrentAddress2 is executed. The function output CurrentAddress2 can be 
executed via the hotkey or a call from the command window. 

Local and Global Scope 

Scope is what variables or functions are visible from a certain location in the code. We will use 
the variable currAddress from the output Current Ad dress as an example of local scope. The currAddress 
variable is only visible from within its function. It cannot be accessed from another function. 

Function declarations are placed in the global scope. Functions can be called from other- 
functions. This includes calling from the command window. Once a function is defined it 
remains in the global scope until we either declare another function with the same name or 
until the IDA session is terminated. Closing an idb file clears out any IDC functions from 
memory. 

Once the outputCurrentAddress is declared we can call it by entering into the command box. 
outputCurrentAddress("some string"); 

We can have our own library of IDC functions load with IDA by adding them to ida.idc 
file. This file is located in the idc directory within the IDA Pro install directory. 


Consider using a custom prefix to avoid name conflicts with functions from 
other scripts. 
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Global Variables 

Auto variables are only in scope for the function they are defined. We need a way to have 
persistent data throughout our script. We need global variables. IDC does not provide direct 
support for global variables. However, global variables can be simulated using arrays. 

Arrays are built in to IDC. An array can contain either string data or a long. The following 
code will create an array and define some items, 
auto gArray; 

gArray = CreateArray("myGlobals") ; 

The code introduces a new IDC function, CreateArray. The prototype for CreateArray is: 

long CreateArray(string name); 

The name must be less than 120 characters. The function will either return the array id 
on success or —1 if the array creation fails. The following code adds some items to the array. 

SetArrayLong(gArray, 23, 415); 

SetArrayString(gArray, 0, "some string data"); 

The prototypes for these new functions are: 

/* 


arguments: 

id 

- 

array 

id 


idx 

- 

index 

of an 

element 

value 

- 

32bit 

value 

to store in the array 

str 

- 

string 

to store in array element 


returns: 1-ok, 0-failed 
*/ 

success SetArrayLong(long id,long idx,long value); 
success SetArrayString(long id,long idx,string str); 

The index idx can be any 32bit number. There is no need to use sequential indexes 
as space is only allocated as it is assigned. The previous example assigns the value 415 to 
index 23 and the string “some string data” to index O.The global data is now assigned 
and can be accessed from anywhere else in the script, other scripts, or through the 
command window. 

In order to access the global data, new IDC functions are introduced. The id of the array 
is needed to access its members. The following code demonstrates the new IDC functions. 

auto arrayld, strltem, longltem; 

arrayld = GetArrayld("myGlobals"); 
strltem = GetArrayElement(AR_STR, id, 0); 
longltem = GetArrayElement(AR_LONG, id, 23); 
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Two new IDC functions are introduced, GetArrayld and Get Array Element. The prototypes 
for the new functions are: 

// get array id by its name 

// arguments: name - name of existing array. 

// returns: -1 - no such array 

// otherwise returns id of the array 

long GetArrayld(string name); 

/* get value of array element 

arguments: tag - tag of array, specifies one of two 

array types: 

#define AR_L0NG 'A' // array of longs 
#define AR_STR 'S' // array of strings 

id - array id 

idx - index of an element 

returns: value of the specified array element. 

note that this function may return char or long 
result. Unexistent array elements give zero as 
a result. 

*/ 

string or long GetArrayElement(long tag,long id,long idx); 

The order of GetArrayElement’s, arguments is different than the SetArray functions. These 
functions can be used anywhere after the array has been defined. The above code snippets do 
not have any error detection for space purposes, but error checks should be added. 

Some IDC libraries of commonly used functions have been released. These libraries 
include global variables among other things and check for errors. One of these, common, idc, is 
written by lallous http://www.openrce.org/downloads/details/81/Common_Scripts. It includes 
other useful functions besides global variables. The following snippet is an example of using 
global variables with the common.idc helper functions. 

First we need to initialize the global variables, using InitGlohalVars, mostly likely this will 
appear in the main function of the script. 

if (InitGlobalVars() == 0) 

{ 

Message("InitGlobalVars() failed\n"); 

} 

Once initialized, we have access to four macro definitions for writing and reading global 
variables. The following are the macros using the same naming convention as used earlier 
(index, value, string). 
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SetGlobalVarLong(index, value) 

SetGlobalVarString(index, string) 

GetGlobalVarLong(index) 

GetGlobalVarString(index) 

Setting array elements using the same data as the previous example: 

SetGlobalVarLong(23, 415) 

SetGlobalVarString(0, "some string data") 

Accessing the items is also much cleaner and simpler. 

auto strltem, longltem; 
strltem = GetGlobalVarString(0) 
longltem = GetGlobalVarLong(23) 

Global variables are very useful as persistent information is often needed. The library 
can be added to idc.idc, allowing access from all scripts. Any functions we find useful can 
be added as well. 


Messages should contain the address of interest as the leftmost item. For 
example: 

Message("%x breakpoint set\n", bpAddr); 

4014c6 breakpoint hit 

This allows the address to be double-clicked taking IDA to the address. 


Simple Script Examples 

So far most of the examples have been using the IDC command window. The command 
window is great for interactive scripting, but it soon becomes unwieldy. Scripts allow us to 
run IDC code without having to re-enter it into a dialog window. 

What are the differences between code snippets and scripts? There aren’t many differ¬ 
ences. All code must reside within functions. Even the command window code was located 
in the _idc function. The following outlines a script. 

#include <idc.idc> 
static some_function() 


} 
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static main() 

{ 

} 

IDC use preprocessor directives like C.The file idc.idc contains IDC function prototypes 
and constants and it is usually included in all scripts. The file also serves as documentation, 
it contains the same information as the help file. IDC supports the #define, #ifdef, and other 
command preprocessor commands. 

The main function is executed by the script. If a main function is not included, the other 
functions will remain in the memory and still be callable. 



Setting up an IDC Development Environment 

There are various things that we can do to facilitate developing IDC scripts. A proper 
text editor is very important. 

Our text editor should support syntax highlighting. Syntax highlighting uses dif¬ 
ferent colors, font, and font sizes to represent different types of data. This allows us to 
easily identify IDC function calls and keywords from other data. Most modern editors 
feature syntax highlighting. Your favorite text editor most likely has an option to add 
new syntaxes. Sebastian Porst posted an IDC syntax file for the Crimson text editor 
http://www.the-interweb.com/serend ipity/exit.php?url_id=157&entry_id=26. 

Along with the proper editor, I change the file extension idc to open with the 
editor. IDA has option to set an editor of your choice for editing IDC scripts. This 
option is available from Options | General | Misc | Editor. 

IDA by default will browse for IDC files from the last location opened. This behavior 
can be changed by editing a configuration file. The cfg directory within the IDA install 
directory holds many configuration files. We are particularly interested in idagui.cfg, 
which has options relating to idag.exe. 

When developing and using IDC files, a commonly changed option is OPEN_ 
DEFAULTJDC_PATH. By default this option is set to NO. Changing the option to YES, 
will always open the IDC file browse dialog in the idc directory. A restart is required. 
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The script from Figure 9.3 will reset a function back to the default color. A coverage tool 
will color basic blocks as it traces execution, similar to Figure 9.4. Other times a user will color 
blocks to highlight certain code. In either case whether between tracing runs or if we are no 
longer interested in certain, we will need to reset the colors. 


Figure 9.3 ResetColor IDC Script 


#include <idc.idc> 
static main(void) 

{ 

auto origEA, currEA, currColor, funcStart, funcEnd; 
origEA = ScreenEA(); 

funcStart = GetFunctionAttr(origEA, FUNCATTR_START); 
funcEnd = GetFunctionAttr(origEA, FUNCATTR_END); 

Message("Welcome to resetColor.idc\n"); 
if (funcStart == -1 || funcEnd == -1) 

{ 

Message("** Error: not in a function **\n"); 
return —1; 

} 

Message("[*] Function: %s\n", GetFunctionName(funcStart) ) ; 

Message("[*] start == Ox%x, end == Ox%x\n", funcStart, funcEnd); 
for (currEA = funcStart; currEA != BADADDR; currEA = 

NextHead(currEA, funcEnd)) 

{ 

if (SetColor(currEA, CIC_ITEM, DEFCOLOR) == 0) 

{ 

Message("** Error: SetColor failed 0x%x **\n", currEA); 

} 

} 

Refresh(); 

Message("resetColor is done\n") ; 

} 
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Figure 9.4 Traced Function 


HHui 



Enter the code from Figure 9.3 into your favorite text editor, preferably a syntax highlight¬ 
ing editor. Save the script with an .idc ending.To run use the menu File | IDC file. ... The 
script will run, returning control to the user. A new window. Recent IDC scripts, will appear 
Figure 9.5.The left button is edit and the right is execute .This allows for quick edit cycles. 
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The code in the script is very similar to some of the snippets in the earlier section. 

The address range of the current function is determined with the GetFuncAttr calls. A loop 
iterates through the function address range and resets the color using a new IDC functions, 
SetColor. 

Figure 9.5 Recent IDC Scripts 

Recent IDC scripts _] 

O cref © cref 
0 ResetColor © ResetColor 


While the script is very simple, it is useful for solving an immediate problem. The next 
section continues this idea while introducing more APIs and concepts. 


Note 

IDA Pro's help file contains documentation for the IDC language. It briefly 
describes constructs such as statements, expressions, and looping. 

The documentation also serves as an API reference for all built in IDC 
functions. 


Writing IDC Scripts 

Scripting languages are very popular because of the immediate results they can provide. 

The user is often trying to solve simple tasks and not developing a full fledged product. 

Scripts can and should be written to automate simple tasks. Complete solutions especially 
in reverse engineering seem to grow organically. Scripting goes hand and hand with this growth. 
Sometimes we write scripts for specific disassembly projects, while other times the scripts can 
be used over and over again. 

Scripting as well plug-in writing does not have to remove the user from the equation. 

In the same sense that IDA is an interactive assembler, scripts should be an interactive tool 
to help the reverse engineer. 

Problem solving with IDC 

This section is an example of how to use IDC to solve a specific problem. It is not by any 
means a complete solution, but rather it demonstrates what can be done with very little 
code and time. 


www.syngress.com 



214 Chapter 9 • IDA Scripting and Plug-ins 


The Problem 

C++ uses indirect calls to call many functions. IDA Pro does not create cross references for 
these functions. 

Problem Background 

C++ reversing presents some new challenges to the reverse engineer. These challenges like 
most in reverse engineering can be solved statically or in runtime. Some recent research 
in static analysis was published by Igor Skochinsky (http://www.openrce.org/articles/full_ 
view/21) (http://www.openrce.org/articles/full_view/23) as a series of articles on the 
OpenRCE website. Some code examples in the form of IDC scripts are also provided. 

A paper has also been published by Paul Vincent Sabanal & Mark Vincent Yason from IBM 
ISS research (https://www.blackhat.com/presentations/bh-dc-07/Sabanal_Yason/Paper/ 
bh-dc-07-Sabanal_Yason-WP.pdf).The paper discusses an internal tool based on IDAPython. 

One of the problems with reversing C++ code relates to indirect calls. Code similar to 
Figure 9.6 is common. A call is made through a register using an offset. It appears that ecx is 
used without being initialized. Ecx is passed to the function and represents the this pointer. IDA 
Pro does not know which function is being called and as such does not create a cross reference. 

Figure 9.6 An Indirect Call 



; Attributes: bp-based frame 

sub_30CBA25 proc near 

arg_0= 

dword ptr 8 

arg_4= 

dword ptr OCh 

arg 8= 

dword ptr 1 0h 

push 

ebp 

mou 

ebp, esp 

mou 

eax, [ecx] 

push 

esi 

mou 

esl, Tebp+arq 8] 

lea 

edx, febp+arq 81 

push 

edx 

push 

[ebp+argb] 

mou 

[ebp+arg 8], esi 

push 

0 

push 

[etop*arg_0] 

call 

dword ptr [eax+14b] 

test 

eax, eax 

jz 

short loc_3 0CBA4E 


If we follow execution using a debugger, we can identify the target function. Checking 
the cross references from the target function will reveal a result similar to Figure 9.7. 
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Figure 9.7 Xrefs to MSF_HB::ReadStream Before the Script 

Ui xrefs to M$F_HB: : Rea d$ti earn (ushort long .void ‘.long *> - a® 


Dir... 1 T Address Text 


UjjUp o text:03015C0C 


dd offset MSF_HB::ReadStream(ushortJong,void “Jong 

1 












OK 


Cancel 


Help 


Search 



Line 1 of 1 


All references are pointers, not function calls. The pointers are part of a V Table. A V Table 
is an array of pointers to functions for a particular object. 

Figure 9.8 MSF_HB::'vftable' 


. text:03015BF8 const MSF_HB:: 'vftable' dd offset 
MSF_HB::QueryImplementationVersion(void) 

. text:03015BF8; DATA XREF: MSF_HB::MSF_HB(void)+9Do 
. text:03015BF8; MSF_HB::~MSF_HB(void)+9Do 


text:03015BFC 

dd 

offset 

MSF 

HB: 

: QueryImplementstionVersion(void) 

text:03015C00 

dd 

offset 

MSF 

HB: 

:GetCbPage(void) 

text:03015C04 

dd 

offset 

MSF 

HB: 

: GetCbStream(ushort) 

text:03015C08 

dd 

offset 

MSF 

HB: 

:GetFreeSn(void) 

text:03015C0C 

dd 

offset 

MSF 

HB: 

:ReadStream(ushort,long, void *,long *) 

text:03015C10 

dd 

offset 

MSF 

HB: 

:ReadStream(ushort,void *,long) 

text:03015C14 

dd 

offset 

MSF 

HB: 

:WriteStream(ushort, long, void *,long) 

text:03015C18 

dd 

offset 

MSF 

HB: 

:ReplaceStream(ushort, void *,long) 

text:03015C1C 

dd 

offset 

MSF 

HB: 

:AppendStream(ushort,void * ,long) 

text:03015C20 

dd 

offset 

MSF 

HB: 

:TruncateStream(ushort, long) 

text:03015C24 

dd 

offset 

MSF 

HB: 

: DeleteStrearn(ushort) 

text:03015C28 

dd 

offset 

MSF 

HB: 

: Commit(void) 

text:03015C2C 

dd 

offset 

MSF 

HB: 

: Close(void) 

text:03015C30 

dd 

offset 

MSF 

HB: 

:GetRawBytes(int(*)(void const *,long)) 

text:03015C34 

dd 

offset 

MSF 

HB: 

: SnMax(void) 

text:03015C38 

dd 

offset 

TM: 

PPdbFrom(void) 


. text:03015C3Cdd offset MSF_HB::CloseStream(ulong) 


When a object method is called, the VTable is accessed and then a call is made using an 
offset to the appropriate function.The code from Figure 9.6 uses this VTable making a call 
to the first MSF_HB:: Read Stream function. 
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Finding VTables for Microsoft binaries is simple if using the Determina PDB 
plug-in by Alexander Sotirov (http://www.determina.com/security.research/ 
utilities/index.html). The pdb contains symbols including VTable names. Finding 
all the VTables can be done with a text search using the following string: 
::'vftable' dd 

Note that the character before vftable is a back tick, whereas the character 
following vftable is a single quote. 


Proposed solution 

In order to find the calling addresses, we could script the debugger and check the stack. 

Prior to IDA 5.2 IDC functionality regarding the debugger was very limited. The functions 
did not allow any handling of debugger events, such as a breakpoint. However, there is a 
workaround using conditional breakpoints. 

This script was written long before 5.2 was available and as such does not rely on the new 
functions.The new functions in 5.2 will be discussed afterwards. Figure 9.9 is the edit breakpoint 
dialog box. The condition can be any IDC statement including a function call. 

Figure 9.9 Setting a Condition to the Handler 
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The function will be called when the breakpoint is hit allowing us to run code during the 
breakpoint. If we don’t want to stop the debugger, the function simply returns O.The function 
evaluates to false allowing execution to continue. The following code can be used to log the 
value of EAX whenever the breakpoint is hit. 

static breakpointHandler() 

{ 

Message("%x bp hit, EAX == Ox%x\n", EIP, EAX); 
return 0; // don't stop on breakpoint 

} 

To check for the caller, we begin by looking at the stack. During a function call, the 
return address is pushed onto the stack. We need the address to the call instruction which is 
one instruction before the return address. After the call in Figure 9.6 the return address 
points to the test instruction rather than the call. This update to the breakpointHandler 
function will log the caller. 

static breakpointHandler() 

{ 

auto caller; 

caller = PrevHead(Dword(ESP) , (Dword(ESP) - 10)); 

Message("%x bp hit, caller == %x\n", EIP, caller); 
return 0; // don't stop on breakpoint 

} 

A new IDC function is introduced, PrevHead. The prototype is: 

long PrevHead (long ea, long minea); 

PrevHead searches for the previous defined instruction or data. The ea argument is the start 
address to begin searching backwards, where minea is the lowest address to include in the 
search. The search in breakpointHandler looks, up to 10 bytes back. Call instructions through reg¬ 
isters are generally only 3 bytes, so the search will find the call. The caller has been determined 
and a cross reference can be added. The updated breakpointHandler adds the cross reference. 

static breakpointHandler() 

{ 

auto caller; 

caller = PrevHead(Dword(ESP), (Dword(ESP) - 10)); 

AddCodeXref(caller, EIP, XREF_USER | fl_CN); 
return 0; // don't stop on breakpoint 

} 

The AddCodeXref adds the cross reference. The prototype is: 

// Flow types (combine with XREF_USER!): 

#define fl CF 16 // Call Far 
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#define fl_CN 17 
#define fl_JF 18 
#define fl_JN 19 
#define fl_F 21 
#define XREF USER 32 


// Call Near 
// Jump Far 
// Jump Near 
// Ordinary flow 

// All user-specified xref types 
// must be combined with this bit 
void AddCodeXref(long From,long To,long flowtype); 


The Message function is removed since it slows down the breakpoint handling. Examining 
other calls in the DLL revealed cross references to be Call Near. The breakpointHandler 
function is complete. 


Figure 9.10 VTable xref Script 


#include <idc.idc> 
static breakpointHandler() 


auto caller; 

caller = PrevHead(Dword(ESP), (Dword(ESP) - 10)); 
AddCodeXref (caller, EIP, XREFJJSER | fl_CN) ; 
return 0; // don't stop on breakpoint 

} 

static setBPs () 


auto currAddr; 
auto vStart; 
auto vEnd; 
auto virFunc; 

Message("setBPs() executed\n"); 
vStart = SelStartO; 
vEnd = SelEndO; 

Message("start = 0x%x\n",vStart); 
Message("end = 0x%x\n",vEnd) ; 

if ((vStart == BADADDR) || (vEnd == BADADDR)) 

{ 

Message("No selection made !!\n"); 
return; 

} 

if ((vStart - vEnd) %4 != 0) 
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Message("not DWORD aligned\n"); 
return; 

} 

for (currAddr = vStart; currAddr < vEnd; currAddr = currAddr + 4) 

{ 

virFunc = Dword(currAddr); 

if (GetBptAttr(virFunc, BPTATTR_EA) == -1) // no bpt there yet 

{ 

if (!AddBptEx(virFunc, 0, BPT_SOFT)) 

{ 

Message("AddBptEx() failed 0x%x\n;", virFunc); 
return; 

} 

if (!SetBptCnd(virFunc, "breakpointHandler()")) 

{ 

Message("SetBptCnd() failed 0x%x\n;", virFunc); 
return; 

} 

Message("BP 0x%x set\n", virFunc); 

} 

else 


Message("BP already set 0x%x\n", virFunc); 

} 

} 

} 

static main () 

{ 

AddHotkey("Alt-f9", "setBPs"); 

} 


Upon running this script, the functions are loaded into memory and main is executed. 
Mains sole purpose is to set a hot key for the setBPs function. 

A VTable similar to Figure 9.8 is selected and then the hotkey is pressed. The setBPs 
function is called by the hotkey. This functions purpose is to set breakpoints on the targets 
found in the VTable. Only one breakpoint can be added per address, thus the function 
checks for the presence of a breakpoint initially. If no breakpoint exists a new software 
breakpoint is added. The breakpoint is then made conditional, with the condition being the 
breakpointHandler function. In this case we choose to return 0 and not stop at the breakpoint. 
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The result after using the script and running the debugger is shown in Figure 9.11, 
which is quite an improvement over the original in Figure 9.7. 

Figure 9.11 Xrefs to MSF_FIB::ReadStream after the Script 

LU xiefs to MSF_HB ReadStieann|us.hort long.void ‘ .long *) □HM 
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Possible Improvements 

If the breakpoint is set on a commonly called function, performance can be degraded. A global 
variable can be used as a counter and the breakpoint could be removed if the counter reaches a 
preset limit. Comments can be added to the calling instruction containing the address of the 
targets. This would allow the user to double click the address to the target function. 

After acquiring the cross reference data, we could analyze it to determine information of 
the classes they represent. Method visibility in C++ can be public, protected, or private. 


Public The target function has at least one call from a function not 

found in any VTable. 

Private The target function is only called from other functions within 

its own VTable. 

Protected The target function is only called from functions located in VTables. 
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If a function is located in more than one V Table at the same offset, it is likely that an 
inheritance relationship exists between the classes. 

The cross reference information could be combined with static analysis to reconstruct 
object models. A graphing tool could be used to represent the models, perhaps in UML. 

New IDC Debugger Functionality 

New IDC functions are added to IDA Pro releases, while it is rare for functions to be deprecated. 
The new functions reflect the new features added to the release. 

IDA 5.2 added 53 functions.The most important additions relate to the scripting of 
the debugger http://www.hex-rays.com/idapro/scriptable.htm.The debugger is now fully 
scriptable from IDC. We can control every aspect of the debugger. This includes acting on 
debugger events, attaching to processes, and tracing. 

A scriptable debugger opens up many possibilities. Unpacking of binaries for a known 
packer is commonly scripted using OllyScript for Ollydbg or in Python for Immunity 
Debugger. Runtime analysis can be fed back into the static analysis. 

The debugger from a plug-in context usually requires a callback to handle events. The IDC 
interface allows for what amounts to a blocking call waiting on events. The function used is 
GetDebuggerEvent and its prototype is: 

long GetDebuggerEvent(long wfne, long timeout); 

The timeout can be set to —1, which is interpreted as infinite. There wfne flags are the 
following: 

WFNE_ANY return the first event 

WFNE_SUSP wait until the process gets suspended 

WFNE_SILENT set: be slient, clear:display modal boxes if necessary 
WFNE_CONT continue from the suspended state 

Most often we want to wait till for a suspended state caused by a breakpoint. The possible 
return values are the following: 

// debugger event codes 

NOTASK process does not exist 

DBG_ERROR error (e.g. network problems) 

DBG_TIMEOUT timeout 

PROCESS_START New process started 

PROCESS_EXIT Process stopped 

THREAD_START New thread started 

THREAD_EXIT Thread stopped 

BREAKPOINT Breakpoint reached 

STEP One instruction executed 

EXCEPTION Exception 

LIBRARY_LOAD New library loaded 
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LIBRARY_UNLOAD 

INFORMATION 

SYSCALL 

WINMESSAGE 

PROCESS_ATTACH 

PROCESS DETACH 


Library unloaded 
User-defined information 
Syscall (not used yet) 

Window message (not used yet) 
Attached to running process 
Detached from process 


This addition to IDC is very welcome as it provides easy scripting use of the debugger. 
A plug-in that works with the debugger is presented later in this chapter. 


Useful IDC Functions 


This section contains a sampling of IDC functions you are likely to see in other scripts and 
while writing new scripts. The functions are grouped into similar categories and include a 
short description of possible usage. 


Reading and Writing Memory 

Reading memory is accomplished through three functions. The functions come in variants 
based on the read size. The functions are Byte, Word, and Dword. 


long 

Byte 

(long 

ea) ; 

// 

get 

a 

byte at 

ea 


long 

Word 

(long 

ea) ; 

// 

get 

a 

word (2 

bytes) 

at ea 

long 

Dword 

(long 

ea) ; 

// 

get 

a 

double-word (4 

bytes) at ea 


The functions return a —1 on failure. In order to differentiate between a failure and a 
value of —1, the macro hasValue should be called. Its prototype is: 

#define hasValue (F) ( (F & FF_IVL) != 0) // any defined value? 

The macro will return a 0 if the value is not defined. 

Writing to memory is accomplished by the Patch family of functions. The functions are 
used for writing within the static analysis within the IDB as well as virtual memory when 
under a debugger. In fact the Patch functions are the only way to modify code during execu¬ 
tion within the debugger. The debugger only allows modification of registers and sections from 
the GUI. Modification to code or data segments requires these IDC functions or a plug-in. 

The function come in three variant based on the write size. The functions are PatchByte, 
Patch Word, and PatchDword. 


void 

PatchByte 

(long 

ea,long 

value) ; 

// 

change 

a 

byte 


void 

PatchWord 

long 

ea,long 

value) ; 

// 

change 

a 

word 

(2 bytes) 

void 

PatchDword 

(long 

ea,long 

value) ; 

// 

change 

a 

dword 

(4 bytes) 


Cross References 

There are different types of cross references, both for data and code. 
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Code Xrefs 

Code cross references are defined by their flowtypes.The following is a list of code 
flowtypes: 


// Flow types ( 

#define fl_CF 16 

#define fl_CN 17 

#define fl_JF 18 

#define fl_JN 19 

#define fl_F 21 

#define XREF USER 32 


combine with XREF_USER!): 

// Call Far 
// Call Near 
/ / Jump Far 
// Jump Near 
/ / Ordinary flow 

// All user-specified xref types 
// must be combined with this bit 


All user created referenced should be combined with XREF_USER. We used fl_CN, call 
near flowtype in the script. There are also near and far jump flowtypes.The ordinary flowtype 
is used between consecutive instructions. 

IDC functions for adding and deleting code cross references: 

void AddCodeXref(long From,long To,long flowtype); 
long DelCodeXref(long From,long To,int undef); 

The undef argument undefmes the To address if this is the last reference to it. 

There are two sets of IDC functions to iterate through references. The difference is in regards 
to recognizing ordinary flows as cross references. The first set will return the ordinary flow first. 


long 

Rfirst 

(long 

From); 

// 

Get 

first 

code 

xref 

from 'From 

long 

Rnext 

(long 

From,long current) 

;// 

Get 

next 

code 

xref 

from 

long 

RfirstB 

(long 

To) ; 

// 

Get 

first 

code 

xref 

to 'To' 

long 

RnextB 

(long 

To,long current); 

// 

Get 

next 

code 

xref 

to 'To' 


The functions consist of a first and a next. Both functions are generally used in a loop to 
iterate through cross references. The following demonstrates these functions: 

auto xfAddr, origAddr; 
origAddr = ScreenEA(); 

xfAddr = RfirstB (origAddr) ; 
while (xfAddr != BADADDR) 


Message("%x to %x, type == %d\n", xfAddr, origAddr, XrefType()); 
xfAddr = RnextB(origAddr, xfAddr); 

} 

The code iterates through all the cross references for the address the cursor is on. It also 
introduces a new IDC function XrefType. The prototype is: 
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long XrefType(void) ; // returns type of the last xref 
// obtained by [RD]first/next[BO] 

// functions. Return values 
// are fl_... or dr_... 

XrefType return the type of the last cross reference accessed. This function also works on 
data cross references which will be discussed shortly. The second set of code cross reference 
functions mirrors the first set. 


long RfirstO (long From); 

long RnextO (long From,long current); 

long RfirstBO (long To); 

long RnextBO (long To,long current); 


These functions do not return ordinary flow cross references. 

Data Xrefs 

The following are valid data types: 


// Data reference 

#define dr_0 1 

#define dr_W 2 

#define dr_R 3 

#define dr_T 4 

#define dr_I 5 

#define XREF USER 32 


types (combine with XREF_USER!): 

// Offset 
// Write 
// Read 

// Text (names in manual operands) 
// Informational 

// All user-specified xref types 
// must be combined with this bit 


Same as code xrefs, user made data xrefs should be combined with XREF_USER. Data xrefs 
have only one set of functions associated with them. 


long 

Dfirst 

(long 

From) ; // Get first 

data 

xref from 'From 

long 

Dnext 

(long 

From,long current); 



long 

DfirstB 

(long 

To); // Get first 

data 

xref to 'To' 

long 

DnextB 

(long 

To,long current); 




Data Representation 

Data representation functions create structures, functions, data, and define code among other 
things. They are the IDC function equivalent to many manual tasks done during disassembly. 
The following is a sampling of these functions. 

success MakeArray(long ea,long nitems); 

success MakeByte(long ea) ; 

long MakeCode(long ea) ; 

success MakeData(long ea, long flags, long size, long tid); 

success MakeDword(long ea); 
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success MakeFunction(long start,long end); 
success MakeStr(long ea,long endea); 

success MakeStructEx(long ea,long size, string strname); 

Comments 

Comments are a key to successful reverse engineering. Comments along with proper nam¬ 
ing are the notes that bind to the binaries we analyze. There are IDC functions to set and 
read comments. 

// repeatable, 0 = standard, 1 = repeatable 
string CommentEx(long ea, long repeatable); 
success MakeComm(long ea,string comment); 
success MakeRptCmt(long ea,string comment); 

long SetBmaskCmt(long enum_id,long bmask,string cmt,long repeatable); 
success SetConstCmt(long const_id,string cmt,long repeatable); 
success SetEnumCmt(long enum_id,string cmt,long repeatable); 
void SetFunctionCmt(long ea, string cmt, long repeatable); 

long SetMemberComment(long id,long member_offset,string comment,long repeatable); 
long SetStrucComment(long id,string comment,long repeatable); 

long GetBmaskCmt(long enum_id,long bmask,long repeatable); 
string GetConstCmt(long const_id,long repeatable); 
string GetEnumCmt(long enum_id,long repeatable); 
string GetFunctionCmt(long ea, long repeatable); 
string GetMarkComment(long slot); 

string GetStrucComment(long id,long repeatable); 

Code Traversal 

IDA has different types of containers for code and data. Some of the containers include 
segments, functions and instruction or data heads. Iterating through different containers 
and areas is very common in scripts. 

Some common iterating functions are: 

long NextAddr(long ea); 

long NextFunction(long ea); 

long NextHead(long ea, long maxea); 

long NextNotTail(long ea); 

long NextSeg(long ea) ; 

long PrevAddr(long ea); 

long PrevFunction(long ea) 

long PrevHead(long ea, long minea); 

long PrevNotTail(long ea) ; 


www.syngress.com 


226 Chapter 9 • IDA Scripting and Plug-ins 


The following code snippet demonstrates some of the iteration functions. 

auto currAddr, func, endSeg,funcName, counter; 

currAddr = ScreenEA(); 
func = SegStart(currAddr) ; 
endSeg = SegEnd(currAddr) ; 

counter = 0; 

while (func != BADADDR && func < endSeg) 

{ 

funcName = GetFunctionName(func) ; 
if (funcName != " ") 

{ 

Message("%x: %s\n", func, funcName); 
counter++; 

} 

func = NextFunction(func); 

} 

Message ("%d functions in segment: %s\n", counter, SegName(currAddr)); 

The script iterates through all the functions belonging to the current segment. The script 
uses the GetFunctionName call to test if an address is in a function. This call returns an empty 
string if the address is not part of a function. Alternatively, GetFunctionFlags could have been 
used. The script prints a list of function addresses along with names for all the functions in 
the segment. The total number of functions is printed upon completion. 

Input and Output 

Thus far the only 1/O used has really been the Message function. There are various input IDC 

functions for different types of data as well as for making selections. 

string AskStr(string defval,string prompt); 

string AskFile(bool forsave,string mask,string prompt); 

long AskAddr(long defval,string prompt); 

long AskLong(long defval,string prompt); 

long AskSeg(long defval,string prompt); 

string Askldent(string defval,string prompt); 

long AskYN(long defval,string prompt); 

The preceding functions retrieve input from the user. The following code snippet 
demonstrates the AskYN IDC function. 

auto answer; 

answer = AskYN(l, "hello"); 
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if (answer == 1) 

Message("YES\n") ; 
else if (answer == 0) 

Message("NO\n") ; 
else 

Message("CANCEL\n"); 

There are IDC functions for file 1/O as well. These file 1/O functions are very similar to 
their C counterparts. 

long fopen(string file,string mode); 
long fseek(long handle,long offset,long origin); 
void fclose(long handle); 
long fgetc(long handle); 

long fprintf (long handle, string format,...); 
long fputc(long byte,long handle); 
long ftell(long handle); 

long writelong(long handle,long dword, long mostfirst); 
long writeshort(long handle,long word,long mostfirst); 
long writestr(long handle,string str); 

long readlong(long handle,long mostfirst); 
long readshort(long handle,long mostfirst); 
string readstr(long handle); 

Basics of IDA Plug-ins 

IDA Pro can be extended through modules. There are various types of modules that can be 
developed for IDA. Plug-ins are one of the types of modules that can be used to extend IDA. 
Sometimes the term plug-in is used incorrectly to cover all extendable modules. 

There are different types of modules available in IDA. The module type is dependent on 
the functionality needed. The categories are: 

■ Plug-in 

■ Toaders 

■ Processor 

■ Debuggers 

Module/Plug-in Resources 

The SDK has many modules/plug-ins with full source code. 

■ Hex-Rays provides an SDK to registered customers. The SDK is included on 
the CD when IDA is purchased and is also available on the Hex-Rays website 
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(http://www.hex-rays.com/idapro/idadown.htm).The SDK is not officially 
supported by Hex-Rays, although the option is available. 

■ The Hex-Rays bulletin board provides help with plug-in issues both by other users 
as well as Ilfak (http://www.hex-rays.com/forum/). 

■ There is not much information available for plug-in development. An excellent 
tutorial was written by Steve Micallef entitled “IDA PTUG-IN WRITING IN 
C/C++” (http://binarypool.com/idapluginwriting/). 

■ Hex-Rays includes various plug-ins with source in the SDK. One of the plug-ins, 

a universal unpacker, is described in a Hex-Rays article (http://www.hex-rays.com/ 
idapro/unpack_pe/unpacking.pdf). 

■ Ilfak has also provided various plug-ins with full source available at his blog 
(http: // hexblog. com). 

■ OpenRCE is a valuable resource for many thing reverse engineering. A portion 
of the download site is dedicated to IDA Pro plug-ins (http://www.openrce.org/ 
downloads/browse/IDA_Plugins). 

Plug-ins are written in C++ and support a variety of compilers and development 
environments. Import libraries are provided for the following: 

■ Visual C++ (32 and 64 bit) 

■ Borland C++ Builder (32 and 64 bit) 

■ GCC C++ Compiler 

■ Windows (32 and 64 bit) 

■ Linux (32 and 64 bit) 

■ Mac OSX (32 and 64 bit) 

This chapter will focus on 32 bit Windows plug-ins using the Microsoft Visual Studio 
2005/2008 compilers. 

Note 

Instructions for other development environments are located in the root 
directory of the SDK. 

install_cb.txt contains CBuilder setup instructions. 
install_mac.txt contains OS X GCC setup instructions. 
install_linux.txt contains Linux GCC setup instructions. 
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Processor modules add support for different CPUs and architectures. Processor modules are 
located in the procs directory. These modules interpret opcodes and generate the disassembly 
we see in IDA. 

Processor modules use the following file extensions: 

■ w32 windows 

■ w64 windows 64 

■ ilx Linux 

■ ilx64 Linux 64 

■ imc OS X 

■ imc64 OS X 64 

Loaders operate similar to operating system loaders. A loader parses executable files, creates 
segments, and determines what segments are code or data. IDA includes loaders for various 
executable files including PE (Portable Executables) and ELF (Executable and Linking Format). 

Loader modules use the following file extensions: 

■ ldw windows 

■ 164 windows 64 

■ llx Linux 

■ 11x64 linux 64 

■ lmc OSX 

■ lmc64 OS X 64 

Debugger modules are complete debuggers are interoperate with IDA. These should not be 
confused with standard plug-ins that work with the built-in debugger modules. Documentation 
for debugger modules consist of source code located in the SDK under \plugins\debugger. 

Standard plug-ins encompass everything else not covered by either the processor or 
loader modules. They are commonly referred to just as plug-ins. These plug-ins operate on 
the disassembly. This is the most common type of plug-in and as such will be documented in 
this chapter. 

Standard plug-ins use the following file extensions: 

■ plw windows 

■ p64 windows 64 

■ plx Linux 

■ plx64 Linux 64 
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■ pmc OS X 

■ pmc64 macosx64 

Introducing the IDA Pro SDK 

Hex-Rays froze the SDK starting with version 4.9. What does this mean to us? Previously 
the SDK changed considerably between different versions. Plug-ins needed to be compiled 
for each SDK as they were not binary compatible. We no longer have to worry about big 
changes between SDKs, besides the addition of new functionality. 

The SDK is on the IDA Pro CD, or is available for download from Hex-Rays (http://www. 
hex-rays.com/idapro/idadown.htm). 

The SDK is a zip file, the latest being idasdk52. I have directory for different SDKs, 
so I extract the archive to \SDK\idasdk52. 


Warning 

The SDK may contain a bug that will prevent proper compilation. The bug is in 
the intel.hpp located in \include. One of the #include listing is wrong. Change 
#include "../idaidp.hpp" 

To 

#include "../module/idaidp.hpp" 

The bug is still present in the latest version 5.2. 


SDK Layout 

The SDK contains various directories. The directories contain, include files, import libraries, 
tools, and source code. The following is an overview of the more important directories: 


include 

All the header files for the SDK. 

Idr 

Source code to several loaders. 

libbor.w32 

Borland for 32 bit Windows plugins. 

libbor.w64 

Borland for 64 bit Windows plugins. 

Iibgcc.w32 

GCC for 32 bit Windows plugins. 

Iibgcc.w64 

GCC for 32 bit Windows plugins. 

Iibgcc32.lnx 

GCC for 32 bit Linux plugins. 

Iibgcc32.mac 

GCC for 32 bit OS X plugins. 

Iibgcc64.lnx 

GCC for 64 bit Linux plugins. 

Iibgcc64.mac 

GCC for 64 bit OS X plugins. 

Iibvc.w32 

Visual Studio for 32 bit Windows plugins. 
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Iibvc.w64 

module 

plugins 


Visual Studio for 64 bit Windows plugins. 
Source code to several processor modules. 
Source code to sample and real plugins. 


Plug-in Syntax 

Plug-ins are loadable libraries, DLL or otherwise, that IDA Pro loads when needed. The 
plug-in has to have a certain structure exported. The structure type depends on the plug-in 
type. This section will cover standard plug-ins as they are the most common type. From now 
on plug-ins will refer to standard plug-ins. Any specifics relating to loaders or processor 
modules will be noted. 

IDA plug-ins are written in C++ and export a plug-in structure, PLUGIN_t. 

plugin_t PLUGIN = 


IDP INTERFACE 

VERSION, 





plugin flags. 

// 

plugin flags 





init. 

// 

initialize 





term. 

// 

terminate, this pointer 

may 

be ] 

NULL. 

run. 

// 

invoke plugin 





comment. 

// 

plugin comment 





help. 

// 

multiline help about 

the 

plugin 


wanted name. 

// 

the preferred short 

name 

of 

the 

plugin 

wanted hotkey 

// 

the preferred hotkey 

to 

run 

the 

plugin 


}; 

The structure contains constants, function pointers, and character string pointers. 

IDP_INTERFACE_VERSION is a constant that will be defined by included SDK files. 
Previous to the 4.9 SDK freeze, this value would be incremented for every new release. 

Since 4.9 this value has remained constant. 

The plugin_flags define how the plug-in operates with IDA. The different flags are described 
in loader.hpp .This field is usually set to 0 or set to PLUGIN_UNL when debugging a plug-in. 

The following three items, init, term, and run are function pointers. 

The init function is the executed when the plug-in is loaded. Its main purpose is to deter¬ 
mine if the plug-in is applicable to the current database. Plug-ins can be specific to processors 
or file formats. Additionally this function could setup the environment for the plug-in once 
run is executed. 

The init function needs to return one of the following: 

■ PLUGIN_SKIP This notifies IDA to not load the plug-in. A plug-in usually 

returns this value, when the architecture or file format isn’t appropriate. For example: 


www.syngress.com 



232 Chapter 9 • IDA Scripting and Plug-ins 

if (inf.filetype != f_PE) 

return PLUGIN_SKIP; // not a PE file 

■ PLUGIN_OK This notifies IDA that the plug-in is appropriate and IDA will load 
the plug-in upon first use. 

■ PLUGIN_KEEP This notifies IDA that the plug-in is appropriate and to leave 
the plug-in in memory. 

The term function is executed when the IDA is being terminated. This function can be used 
to clean up resources used during the plug-ins lifetime. Many plug-ins set this pointer to NULL. 

The run function is executed by running the plug-in. This function accepts arguments. 
The arguments are defined within the plugin.cfg file located in the plugin directory. Many 
plug-ins use the run function to do all the necessary work. Other plug-ins use the run 
function to set up callbacks. Debugging functionality in the SDK is handled by callbacks. 

■ Comment is pointer to a short character string description for the plug-in. 

■ Help is also a pointer to a character string. However unlike the Comment string, 
Help is usually a multiline description of the plug-in. 

■ Wanted_name is the name that is displayed in the plug-in list accessible from 
(File | Edit | Plugins). 

■ Wanted hotkey sets up a hotkey to run the plug-in. This hotkey can be 
overridden via plugins.cfg. 

Currently the comment and help fields are not used by IDA, but this may change in the future. 

Setting up the Development Environment 

This section covers setting up the development environment under Visual Studio 2005 and 
2008. Build instructions for other platforms are available in the base directory of the SDK. 

Setting up development environments can be tedious. The easiest way to start writing 
plug-ins is to use the IDA Pro Plug-in Wizard. The wizard is compatible with Visual Studio 
2005 and 2008. All appropriate compiler and linker options will be configured by the 
wizard. 

The IDA Pro Plug-in Wizard is available from http://ringzero.net/re.The wizard is 
compatible with: 

■ Visual Studio 2008 

■ Visual Studio 2005 

■ Visual C++ 2008 Express Edition 

■ Visual C++ 2005 Express Edition 
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Tools & Traps... 


Building Plug-ins under Linux 

Setting up a proper plug-in build environment under Linux can be complicated. The 
following makefile can be used to build plug-ins. Note that command lines must begin 
with a tab. The lines following the all, install, and clean labels are command lines. 

# Makefile for IDA Pro Plugins under Linux 

# Updated version of makefile from Steve Micallef' s 

# IDA Plugin Writing Tutorial 

# http : //www.binarypool.com/idapluginwriting/ 

# Set your plugin name here. PLUGINNAME.plx 
PLUGINNAME=myplugin 

# Set your IDA install directory 
IDABASEDIR=/usr/local/idaadv 

# Set your IDA SDK directory 
SDKBASEDIR=/usr/local/idaadv/sdk 

# Compiles all cpp files in current dir 
SRC=$(wildcard *.cpp) 

OBJS=$(SRC:.cpp—.o) 

CC=g++ 

LD=g++ 

CFLAGS=-D_IDP_ -D_PLUGIN_ -c -D_LINUX_\ 

-1$(SDKBASEDIR)/include $(SRC) 

LDFLAGS=-shared $(OBJS) -L$(IDABASEDIR) -lida -no-undefined \ 

-Wl,-version-script=$(SDKBASEDIR)/plugins/plugin.script 

all: 

$(CC) $ (CFLAGS) 

$(LD) $(LDFLAGS) -o $(PLUGINNAME).plx 
install: 

cp $ (PLUGINNAME) .plx $(IDABASEDIR)/plugins 

Continuec 
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clean: 

-rm -f *.plx *.o core 
rebuild: clean all 


Simple Plug-in Examples 

Now that we have setup a development environment, we can move on to writing some 
plug-ins. We will first build a simple “hello world” plug-in. This will allow us to test our 
environment and verify that IDA is properly loading and executing our plug-in. The find 
memcpy plug-in will demonstrate some of the IDA API including instruction decoding and 
some UI code. 

The Hello World Plug-in 

Start Visual Studio and select the IDA Pro Plugin Wizard. Enter the project name and click OK. 
Figure 9.12 shows the selection within Visual C++ 2008 Express Edition. 
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The wizard is shown in Figure 9.13.The plug-in type will default to plug-in, which is what 
we are building. The Name of Author field is optional, but will appear in the header comments if 
present. 

The SDK Path is required to build the plug-in. Use the button to bring up the folder- 
browse dialog. Be sure to select the base on the SDK directory. 

The final item is optional but very useful. A post build event is created in the project 
properties. The event copies the plug-in to the appropriate IDA Pro directory. Use the button 
to bring up the folder browse dialog. Be sure to select the base on the IDA Pro install 
directory. 

The paths only need to be filled in once as the wizard saves the options. 

Figure 9.13 IDA Pro Plugin Wizard Dialog 
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Click Finish and the wizard will complete preparing the project. The IDA Pro 
Plug-in wizard will have a sample template. The following code can be copied over the 
template. The key combination to start a build varies on configuration, but by default 

it is CRTL+SHIFT+B. 

#include <ida.hpp> 

#include <idp.hpp> 

#include <loader.hpp> 

// Determine if the plugin is suitable. Return: 

// PLUGIN_SKIP - plugin not suitable, wont be used 
// PLUGIN_KEEP - plugin is suitable, keep in memory 

// PLUGIN_OK - plugin is suitable, load when used 
int init(void) 

{ 

return PLUGIN_OK; 

} 

// plugin termination function. Unhook any notification points, 
void term(void) 

{ 

return; 

} 

// This function is called when the plugin is executed. 

// The arg is configured in the plugin, cfg file, 
void run(int arg) 

{ 

msg("Hello world! my address is %a\n", get_screen_ea()); 
return; 

} 

char comment[] = "hello world"; 
char help[] = "hello world"; 

// Name of plugin in ( Edit | Plugins ) 

// An entry in plugins.cfg can overide this field, 
char wanted_name[] = "hello world"; 

// Plugin's hotkey 

// An entry in plugins.cfg can overide this field, 
char wanted_hotkey[ ] = 
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// PLUGIN DESCRIPTION BLOCK 
plugin_t PLUGIN = 

{ 

IDP_INTERFACE_VERSION, 

PLUGIN_UNL, // plugin flags 

init, // initialize 

term, // terminate, this pointer may be NULL, 

run, // invoke plugin 

comment, // comment about the plugin 

help, // multiline help about the plugin 

wanted_name, // the preferred short name of the plugin 

wanted_hotkey // the preferred hotkey to run the plugin 

}; 

The previous code is the equivalent of a hello world program. It implements the key 
functions outlined in the pluginjt structure. The plug-in uses a flag, PLUGIN_UNL. This flag 
is often used for debugging plug-ins. It is defined as: 

#define PLUGIN_UNL 0x0008 // Unload the plugin immediately after 

// calling 'run'. 

// This flag may be set anytime. 

// The kernel checks it after each 
// call to 'run' 

// The main purpose of this flag is to ease 
// the debugging of new plugins. 

The plug-in will be unload after run is executed. This allows us to make changes, 
recompile, and copy the plug-in to the plug-in directory. If the flag is not set, IDA needs to 
be restarted as it retains an open file handle to the plug-in. A workaround will be presented 
shortly. Note that unloading only occurs after executing run. If the init function returns 
PLUGIN_KEEP, the plug-in remains in memory and will not be unloaded until the run 
function is executed. However if the init function returned PLUGIN_OK, plug-in is only 
loaded upon first use. 

Warning 


If the init function sets callbacks it must return PLUGIN_KEEP. Otherwise, the 
memory addresses used may become invalid, as the plug-in may load at a 
different address. 
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The plug-in’s run function outputs a message containing “hello world” and the current 
address.The IDA API call used is get_screen_ea.Th\s function is equivalent to the IDC function 
used earlier in this chapter, ScreenEA. The IDA API contains equivalents to many IDC functions 
as well as functionality not available from IDC. The hello world plug-in verifies that we have a 
working development environment. 

The find memcpy Plug-in 

With a working development environment we can move on to more useful plug-ins, while 
introducing some new IDA API calls. This plug-in searches for inline memcpy $ (See Figure 9.14). 
Compilers often inline library calls as an optimization. Memcpy is commonly inlined along 
with string functions such as strlen and strcpy. 

The assembly in Figure 9.14 uses movsd and movsb instructions to copy data.The movs 
instructions operate on the edi and esi register. Esi holds the source address, while edi points 
to the destination. Movsd copies a dword (four bytes) and movsb copies a single byte. 

rep is a prefix that repeats the movs instructions. Every time movs instruction executes ecx 
is decremented.The movs instructions stop once ecx reaches zero. (See Figure 9.15). 


Figure 9.14 Inline memcpy 



; memcpy 

(edi. 

esi, eax) 

.text:00418ECC 

mov 

ecx. 

eax 

; copy 

eax into ecx 

.text:00418ECE 

shr 

ecx. 

2 

; shift 

ecx right by 2 (divide by 4) 

.text:00418ECE 




; ecx = 

number of dwords to copy 

.text:00418ED1 

rep 

movsd 


; copy 

dwords from esi to edi 

.text:00418ED3 

mov 

ecx. 

eax 

; copy 

eax into ecx 

.text:00418ED5 

and 

ecx. 

3 

; and ecx by 3 

.text:00418ED5 




; ecx = 

number of bytes to copy 

.text:00418ED5 




; (remaining bytes) 

.text:00418ED8 

rep 

movsb 


; copy 

bytes from esi to edi 
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Figure 9.15 rep movsd flowchart 



Eax contains the number of bytes to copy. The shr (shift right) instruction divides ecx by 
four, calculating the number of dwords to copy The rep movsd instruction copies ecx dwords 
from esi to edi. After copying the dwords, there can be zero to three bytes left to copy. The and 
instruction calculates the remaining bytes which rep movsb copies. 

The plug-in in Figure 9.16 illustrates a method for finding these types of code constructs. 

Figure 9.16 Find memcpy Plug-in 

/-k-k-k-k-k'k-k-k'k-k-k-k-k-k-k'k-k-k'k-k-k'k-k-k'k-k-k-k-k-k-k'k-k-k-k-k-k-k-k'k-k-k-k-k-k'k-k'k-k-k'k-k-k'k-k-k'k-k-k-k-k-k-k'k'k-k 

* Find memcpy() IDA Pro plugin 

* 

* Copyright (c) 2008 Luis Miras 

* Licensed under the BSD License 
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* Requirements: The plugin requires x86 processor. 

* Description: The plugin searches for rep movsd/rep movsb 

* pairs identidying them as memcpyO 

* Single rep movsd and rep movsb instructions 

* are also recorded 

* Data structures: a netnode is the main data structure. 

* movsobj_t represents the either pairs 

* or single instructions. 

* 

* netnodes are implemented internally as B-trees. 

* IDA uses netnodes extensively for its own storage. 

* netnodes are defined in netnode.hpp. 

* 

* netnodes in the plugin: calls - holds all indirect calls 

* vtable - holds all vtables 

* netnodes have various internal data structures. 

* The plugin uses 2 types of arrays: 

* altval - a sparce array of 32 bit values, initially set to 0. 

* supval - an array of variable sized objects (MAXSPECSIZE) 

* The plugin holds base addresses in altval and movsobj_t objects 

* in supval 

*************-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k j 

#include <ida.hpp> 

#include <idp.hpp> 

#include <loader.hpp> 

#include <allins.hpp> 

#include cintel.hpp> 

#define NODE_COUNT -1 

struct movsObj { 

ea_t movsDW; // addr of rep movsd. BADADDR if none 
ea_t movsBT; // addr of rep movsd. BADADDR if none 

} ; 

typedef movsObj movsobj_t; 
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static const char* header[] = {"Address", "Type", "Movsd/b distance"}; 
static const int widths[] = { 16, 25, 25}; 
char window_title[] = "Inline memcpy" ; 

/★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★ 

* Function: processMemcpy 

* 

* This function determines the types of memcpy based on the movsobj_t 

* and calculates distance between rep movsd and rep movsb 

***********************************************************************j 

char* processMemcpy(movsobj_t* my_movs, ea_t* movs_distance) 

{ 

if (my_movs->movsDW == BADADDR) 

{ 

*movs_distance = BADADDR; 
return "memcpy movsb only"; 

} 

else if (my_movs-> movsBT == BADADDR) 

{ 

*movs_distance = BADADDR; 
return "memcpy movsd only"; 

} 

else 

{ 

*movs_distance = my_movs-> movsBT - my_movs->movsDW; 
return "memcpy ()"; 

} 

} 

/************************************************************************* 

* Function: description 

* 

* This is a standard callback in the choose2() SDK call. This function 

* fills in all column content for a specific line. Headers names are 

* set during the first call to this function, when n == 0. 

* arrptr is a char* array to the column content for a line. 

* arrptr[number of columns] 

* 

* description creates 3 columns based on the header array 

*************************************************************************j 
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void idaapi description(void *obj,ulong n,char * const *arrptr) 

{ 

netnode *node = (netnode *)obj; 
movsobj_t my_movs; 
char* outstring = NULL; 
ea_t movs_distance; 

if ( n == 0 ) // sets up headers 

{ 

for ( int i=0; i < qnumber(header) ; i++ ) 
qstrncpy(arrptr[i] , header[i], MAXSTR); 
return; 

} 

// list empty? 

if (!node->altval(NODE_COUNT)) 
return; 

node->supval(n-1, &my_movs,sizeof(my_movs)); 
outstring = processMemcpy(&my_movs, &movs_distance) ; 
qsnprintf(arrptr[0] , MAXSTR, "%08a", node->altval(n-1) ); 
qsnprintf(arrptr[1] , MAXSTR, "%s", outstring); 

if (movs_distance != BADADDR) 

{ 

qsnprintf(arrptr[2], MAXSTR, "%02x", movs_distance); 

} 

else 

{ 

qsnprintf(arrptr[2] , MAXSTR, ""); 

} 

return; 

} 

/•k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k 

* Function: enter 

k 

* This is a standard callback in the choose2() SDK call. This function 

* is called when the user pressed Enter or Double-Clicks on a line in 

* the chooser list. 

-kk-k-k-k-k-k-kk-k-k-k-k-k-k-kkk-k-k-k-k-k-kk-k-kk-k-kk-kkk-k-k-k-k-k-kkk-kkk-kk-k-kk-kk-k-k-k-k-k-k-kkk-kk-k-kk-kkk-k-k-k-k I 

void idaapi enter(void * obj,ulong n) 
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{ 

ea_t addr; 

netnode *node = (netnode *)obj; 
addr = node->altval(n-1) ; 
jumpto(addr) ; 
return; 

} 

/-k-k-kkk-kk-k-kk-kkk-kk-k-kk-kk-k-kk-k-kk-kk-k-kk-k-kk-k-k-k-k-kk-k-k-k-k-k-kk-k-kk-k-kk-kk-k-kk-k-kk-kkk-kk-k-kk-k-kk-k 

* Function: destroy 

k 

* This is a standard callback in the choose2() SDK call. This function 

* is called when the chooser list is being destroyed. Resource cleanup 

* is common in this function. The netnode deleted here. 

kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk/ 

void idaapi destroy(void* obj) 

{ 

netnode *node = (netnode *)obj; 

node->kill() ; 

return; 

} 

/kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 

* Function: size 

k 

* This is a standard callback in the choose2() SDK call. This function 

* returns the number of lines to be used in the chooser list. 

k-kk-k-kk-k-k-k-k-kkk-kkk-k-kk-k-kk-k-k-k-k-kkk-kkk-k-kk-k-kk-kkk-k-k-k-kkkkkkk-k-k-k-k-kk-k-k-k-k-k-kk-k-kk-k-kk-kk-kJ 

ulong idaapi size(void* obj) 

{ 

ulong mysize; 

netnode *node = (netnode *)obj; 
mysize = node->altval(NODE_COUNT); 
return mysize; 

} 

j k-kkkkk-kkk-kkk-kk-kkk-kkk-kkkkk-kkk-kkkkk-kkkkkk-kk-kkk-kkk-kk-kkk-kkk-kkkkkkkk-kk-kkk-kk-k-kk 

* Function: functionSearch 

* 

* functionSearch looks through functions for rep movsd and rep movsb 

* memcpy is defined as a rep movsd followed by rep movsb 

* single rep movsd and movsb are also recorded 
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* 

* last_movs is used to track for rep movsd/rep movsb sets 

* the netnode's alval and supval arrays are used 

* node->alset contains the base address 

* node->supset contains a movsobj_t object 

* 

* memcpy() == movsobj_t with {addr, addr} 

* mosvd only == movsobj_t with {addr, BADADDR} 

* movsb only == movsobj_t with {BADADDR, addr} 

* NOTE: this function misses rep movw (66 F3 A5) instructions 

***********************-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k J 

void functionSearch(func_t* funcAddr, netnode* node) 

{ 

movsobj_t my_movs; 

int counter = node->altval(NODE_COUNT) ; 
ea_t last_movs = BADADDR; 
ea_t addr = funcAddr->startEA; 

while (addr != BADADDR) 


flags_t flags = getFlags(addr); 
if (isHead (flags) && isCode (flags) ) 

{ 

// fill cmd, only looking for 2 byte instructions 
if (ua_anaO(addr) == 2) 

{ 

if ((cmd.auxpref & aux_rep) && (cmd.itype == NN_movs)) 

{ 

if (cmd.Operands[1].dtyp == dt_dword) // rep movsd 

{ 

if (last_movs != BADADDR) 

{ 

// two consecutive rep movsd 
// set the previous one to movsd only 
my_movs. movsDW = last_movs; 
my_movs. movsBT = BADADDR; 
node->altset(counter, last_movs); 

node->supset(counter++, &my_movs,sizeof(my_movs)) 

} 
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// found a rep movsd waiting for rep movsb 
last_movs = cmd.ea; 

} 

else if (cmd.Operands[1].dtyp == dt_byte) // rep movsb 

{ 

if (last_movs == BADADDR) 

{ 

// rep movsb with no preceding rep movsd 
my_movs. movsDW = BADADDR; 
my_movs. movsBT = cmd.ea; 
node->altset(counter, cmd.ea); 

node->supset(counter++,&my_movs,sizeof(my_movs)); 

} 

else // memcpyO 

{ 

// complete set rep movsd/rep movsb 
my_movs. movsDW = last_movs; 
my_movs. movsBT = cmd.ea; 
node->altset(counter, last_movs); 

node->supset(counter++, &my_movs,sizeof(my_movs)); 

} 

last_movs = BADADDR; 

} 

else 


msg("%x: rep", addr); 
msg("ERROR !!!\n"); 

} 

} 

} 

} 

addr = next_head(addr, funcAddr->endEA); 

} 

if (last_movs != BADADDR) 

{ 

// a remaining single rep movsd 
my_movs. movsDW = last_movs; 
my_movs. movsBT = BADADDR; 
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node->altset(counter, last_movs); 

node->supset(counter++, &my_movs, sizeof(my_movs)); 

} 

node->altset(NODE_COUNT, counter); 
return; 

} 

/************************************************************************* 

* Function: collectData 

* This function iterates through all functions calling functionSearch 

/************************************************************************* 
void collectData(netnode* node) 

{ 

for (uint i = 0; i < get_func_qty() ; ++i) 

{ 

func_t *f = getn_func(i); 
functionSearch(f, node); 

} 

return; 

} 

/************************************************************************** 
Function: init 

* 

* init is a plugin_t function. It is executed when the plugin is 

* initially loaded by IDA 

************************************************************************* I 

int init(void) 

{ 

// plugin only works for x86 executables 
if (ph.id != PLFM_386) 
return PLUGIN_SKIP; 
return PLUGINJDK; 

} 

I************************************************************************* 

* Function: term 

* 

* term is a plugin_t function. It is executed when the plugin is 

* unloading. Typically cleanup code is executed here. 

* The window is closed to remove the choose2() callbacks 

*************************************************************************f 
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void term(void) 

{ 

close_chooser(window_title); 
return; 

} 

/******************-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k 

* Function: run 

~k 

* run is a plugin_t function. It is executed when the plugin is run. 

* This function collects data and and displays results 

* 

* arg - defaults to 0. It can be set by a plugins.cfg entry. In this 

* case the arg is used for debugging/development purposes 


* ;plugin displayed name 

filename 

hotkey 

arg 

* find memcpy 

findMemcpy 

Ctrl-F12 

0 

* find memcpy unload 

findMemcpy 

Shift-F12 

415 


~k 


* Thus Shift-F12 runs the plugin with an option that will unload it. 

* This allows (edit/recompile/copy) cycles. 

*************************************************************************\ 
void run(int arg) 

{ 

char node_name[] = "$ inline memcpy"; 

if(arg == 415) 

{ 

PLUGIN.flags |= PLUGINJJNL; 
msg ("Unloading plugin...\n") ; 
return; 

} 

netnode* node = new netnode; 
if(close_chooser(window_title)) 

{ 

//window existed and is now closed 
msg("window existed and is now closed\n"); 

} 

if (node->create(node_name) == 0) 

{ 

msg("ERROR: creating netnode %\n", node_name); 
return; 

} 
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// set netnode count to 0 
node->altset(NODE_COUNT, 0) ; 

// look for memcpys 
collectData(node) ; 


// create chooser list 


choose2(false, // 

- 1 , - 1 , - 1 , - 1 , // 
node, // 

qnumber(header), // 
widths, // 

size, // 

description, // 

window_title, // 
- 1 , // 

0 , // 

NULL, // 

NULL, // 

NULL, // 

NULL, // 

enter, // 

destroy, // 

NULL, // 

NULL); // 


box 

non-modal window 

position is determined by Windows 
object to show 
number of columns 
widths of columns 

function that returns number of lines 
function that generates a line 
window title 

use the default icon for the window 

position the cursor on the first line 

"kill" callback 

"new" callback 

"update" callback 

"edit" callback 

function to call when the user pressed Enter 
function to call when the window is closed 
use default popup menu items 
use the same icon for all line 


return; 

} 

char comment!] = "findMemcpy - finds inline memcpy"; 
char help[] = "findMemcpy\n" 

"This plugin looks through all functions\n 
"for inline memcpy\n"; 
char wanted_name[] = "findMemcpy"; 
char wanted_hotkey[] = 


/* defines the plugins interface to IDA */ 
plugin_t PLUGIN = 


IDP_INTERFACE_VERSION, 

0, // plugin flags 

init, // initialize 

term, // terminate, this pointer may be NULL. 
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run. 

// 

invoke plugin 






comment. 

// 

comment about 

the plugin 




help. 

// 

multiline help 

about 

the 

plugin 


wanted name. 

// 

the preferred 

short 

name 

of 

the 

plugin 

wanted hotkey 

// 

the preferred 

hotkey 

to 

run 

the 

plugin 


Compile the plug-in and run the plug-in. No longer are we bound to the message window, 
findMemcpy opens a chooser list box similar to Figure 9.17. All the functionality of a built in list 
box is provided. The list can be sorted by any of the columns. Clicking on a line will jump to 
the memory address in the disassembly view. 

Figure 9.17 Find Memcpy Results 

^ IDA - C.\ieverse\dbghelp dlt\dbghelp.idb (dbghelp.dll) - [Inline memcpy] _ DIM ! 

File Edit dump Search View Debugger Options Windows Help _ 5 1 X 

1 e 3 H11 » - 11 I ft [ 1 I ltext_ hi hll J 11 ~ ♦ * 11 B B CD 1 P «| 

1 a a ♦!! m @ y :< a a / a| [ a eg n la ■■■■•■II * T illZ ~~H8S8“ I | a 

j a En 11 s r; r iS' v » N x|l SS « v SM k - /|j ; » ffll lU ll| n|| & A A &| 

Jj] IDA Vi ew-A l| § HexView-A Inline memcpy 


▼ Address 

Type 

Movsd/b distance 

A 

I030BCD80 

memcpyO 

Oa i] 


030BF345 

memcpyO 

07 


030BF921 

memcpyO 

07 


030C249F 

memcpyO 

07 


030C2B05 

memcpyO 

07 


030C32C2 

memcpyO 

07 


030C342A 

memcpyO 

07 


030C376E 

memcpyO 

07 


030C3806 

memcpyO 

09 


030C3881 

memcpyO 

07 


030C3A3F 

030C40EA 

memcpy movsd only 
memcpyO 

0a 


030C4C78 

030C6535 

memcpy movsd only 
memcpyO 

07 


030C6B4D 

030C8CB4 

memcpy movsd only 
memcpyO 

07 


030C8E11 

memcpyO 

07 


030CA246 

memcpyO 

09 


030CA865 

030CA9E5 

030CB860 

memcpy movsd only 
memcpy movsd only 
memcpyO 

07 


030CB8B2 

memcpyO 

07 


030CB916 

030CBDB5 

memcpy movsd only 
memcpyO 

0a 


030CCGAB 

memcpyO 

07 


030CC6DE 

memcpyO 

07 


030CC861 

memcpyO 

Od 


030CD234 

memcpyO 

07 


030CD857 

030CDB49 

memcpy movsd only 
memcpyO 

07 


030CDD1D 

030CDECE 

memcpy movsd only 
memcpyO 

07 


030CE10B 

mnrrwfi 

memcpy movsd only 

mpmi'nu mn\/«rl nnln 


V 

Line 257 of 332 

Retrieving information from the database AU: idle Down 

Disk: 4GB 



www.syngress.com 










250 Chapter 9 • IDA Scripting and Plug-ins 


The plug-in introduce some new IDA API functionality, the list box being the most 
apparent. However the plug-in also introduces one of IDA’s built data types. IDA uses the 
netnode class to store internal information. 

What is a netnode? Netnode is defined in netnode. hpp and is internally implemented 
as a B-Tree. Netnode are saved with the database and thus can provide permanent storage 
tied to an idb. This plug-in kills the netnode, since it doesn’t require permanence. Two types 
within netnode are used in both this plug-in and the indirectCall plug-in presented later in 
the chapter. The types are altval and supval. 


Type Description 


altvals This is a sparse array holding 32 bit values, altvals is often used 

with addresses as keys. The value bound to the key is then used 
as an index to the supval array. 

supvals This is an array of variable sized objects (up to MAXSPECSIZE 

defined as 1024 bytes). 


The plug-in uses the supval array to store movsobjjt objects. Each movsobjjt represents 
either a memcpy or a partial memcpy. A partial memcpy would be a single rep movsd or rep movsb. 

struct movsObj { 

ea_t movsDW; // addr of rep movsd. BADADDR if none 

ea_t movsBT; // addr of rep movsd. BADADDR if none 

} ; 

typedef movsObj movsobj_t; 

If a single or unmatched rep movsd is encountered the missing item’s address is recorded 
as BADADDR. Altval is used as a standard array and the value is the base address to be 
displayed. The base address is rep movsd’s address except for a single rep movsb. 

Altval holds the array count at index NODE_COUNT (~1). Holding the array count at 
index —1 is common among other plug-ins as well. 

Having covered the data types we can move to the run function. The netnode is created 
in this function. The use of a “$’ prefix to the netnode name is recommended in netnode. 
hpp. Tocation names should not be used as IDA names netnodes by location name. 

This plug-in uses arguments to the run function. Arguments are defined in the plugins, cfg 
file located in the plugin directory. Add the following to the end of the plugins.cfg file: 

find_memcpy findMemcpy Ctrl-F12 0 

find_memcpy_unload findMemcpy Shift-F12 415 
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Since the plug-in uses callbacks it cannot unload itself after executing run. The extra 
option is added in order to unload the plug-in. If the proper argument is received the 
plug-in flags are modified allowing the plug-in to unload. Unloading allows us to compile 
and copy over a new version of the plug-in without having to shutdown and restart IDA. 
The section. Plug-in “Debugging Strategies”, contains more debugging techniques, 
if(arg == 415) 


PLUG IN. flags |= PLUGIN_UNL; 
msg ("Unloading plugin...\n") ; 
return; 

} 

Collecting Data 

The rest of the run calls two functions, collectData and the choose2 API function which creates 
the list box. The collectData function iterates through all the functions calling functionSearch, 
but not before introducing two new API calls. The new functions are defined in funcs.hpp. 

// Get pointer to function structure by number 

// n - number of function, is in range 0 . .get_func_qty()-1 

// Returns ptr to a function or NULL 

idaman func_t *ida_export getn_func(size_t n) ; 

// Get total number of functions in the program 
idaman size_t ida_export get_func_qty(void) ; 

The real work is done by functionSearch. The function may look similar to what we saw 
in IDC. Iterating over areas is very common in scripts and plug-ins. The while loop iterates 
over all the defined items in the function. 

// Get start of next defined item. Return BADADDR if none exist. 

// maxea is not included in the search range 

idaman ea_t ida_export next_head(ea_t ea, ea_t maxea); 

The first if statement defines that we are looking for instructions. The next if statement 
includes a new API call. 

// Analyze the specified address and fill 'cmd' 

// This function does not modify the database 
// Returns the length of the (possible) instruction or 0 
idaman int ida_export ua_anaO(ea_t ea); 

The ua_anaO function part of a family of functions that analyzes instructions defined in 
ua.hpp. The ua__anaO is the most minimal as it only analyzes the address without modifying 
the database. The analysis goes into ‘cmd’, which holds instruction information. 
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idaman insn_t ida_export_data cmd; // current instruction 

The insn_t type is actually a class which holds both generic and processor specific 
instruction information. Both rep movsd and rep movsb are two byte instruction satisfying 
the if statement. The following lines use various cmd attributes. 

01 if ((cmd.auxpref & aux_rep) && (cmd.itype == NN_movs)) 

02 { 

03 if (cmd.Operands[1].dtyp == dt_dword) // rep movsd 
04 { 

05 // removed for now 

06 } 

07 else if (cmd.Operands[1].dtyp == dt_byte) // rep movsb 

08 { 

09 // removed for now 

Line 1 looks at both the auxpref and itype attribute for the analyzed instruction. The itype 
is represents the instruction mnemonic. 

ushort itype; // instruction code (see ins.hpp) 

The file ins.hpp is not used and instruction nmemonics are located in allins.hpp. The 
instruction nmemonics are stored in very large enums. The first two characters define the 
processor. The auxref attribute is a processor dependent field and as such aux_rep is bit flag 
defined in intel.hpp. Line 1 thus looks for rep movsd or rep movsb. 


Note 

What about rep movsw? Intel has these wonderful things called prefixes that 
can change the sizing of instruction that follows it. 

F3 A5 rep movsd 

66 F3 A5 rep movsw 

F3 A4 rep movsb 

The 0x66 prefix is what separates rep movsd from rep movsw. The plug-in 
will miss rep movsw instructions. 


The if statements in lines 3 and 7 look at the operands of the instruction. The insnjt class 
contains an array of op_t objects which are the operands. 

#define UA_MAXOP 6 

op_t Operands[UA_MAXOP] ; 
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The operand class provides more detail about instructions. Using operands you could 
determine which register an instruction was using, or if the instruction had an immediate 
as an offset. The attribute the code uses is dtype. 

#define dt_byte 0 // 8 bit 

#define dt_word 1 // 16 bit 

#define dt_dword 2 // 32 bit 

The instruction is now sufficiently decoded to determine rep movsd or rep movsb. The rest of 
the code in funcSearch looks for matched pairs of rep movsd /rep movsb. The variable Iast_bp is used 
to track the order. Any unmatched moves are also stored. The plug-in does not perform any 
code analysis and it is possible that there are jumps connecting unmatched sets. The difference 
between rep movsd and rep movsb is also calculated in an effort to spot discrepancies. 


Displaying Data 

IDA has API functions for both single and multi-column list boxes. The choose2 function is a 
wrapper that calls choose with preset options such as creating a modal window. The function 
works well as is relatively easy to use. 

The following is the commented prototype from kernwin.hpp. 


inline ulong choose2( 
void *obj, 
int width, 

ulong (idaapi*sizer)(void *obj), 

char *(idaapi*getl)(void *obj, 
ulong n,char *buf), 

const char *title, 

int icon, 

ulong deflt=l, 
chooser_cb_t *del=NULL, 


void (idaapi*ins)(void *obj)=NULL, 
chooser_cb_t *update=NULL, 


// object to show 
// Max width of lines 
// Number of items 

// Description of 
// n-th item (l..n) 

// 0-th item if header line 

// menu title (includes ptr to 
help) 

// number of the default icon to 
display 

// starting item 

// cb for "Delete" (may be NULL) 
// supports multi-selection 
scenario too 

// returns: 1-ok, 0-failed 
// cb for "New" (may be NULL) 

// cb for "Update"(may be NULL) 
// update the whole list 

// returns the new location of 
item 'n' 
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} 


void (idaapi*edit)(void *obj,ulong n)=NULL, 
void (idaapi*enter)(void * obj,ulong n)=NULL, 
void (idaapi*destroy)(void *obj)=NULL, 
const char * const *popup_names=NULL, 
int (idaapi*get_icon)(void *obj,ulong n)=NULL); 


// cb for "Edit" 

// (may be NULL) 

// cb for non-modal 

"Enter" (may be NULL) 

// cb to call when the 
window is closed (may be NULL) 

// Default: 

// insert, delete, edit, refresh 
// cb for get_icon 
// (may be NULL) 


The following is the call to choose2 from find memcpy. 


// create chooser list 
choose2(false, 

- 1 , - 1 , - 1 , - 1 , 
node, 

qnumber(header) , 

widths, 

size, 

description, 

window_title, 

- 1 , 

0 , 

NULL, 

NULL, 

NULL, 

NULL, 

enter, 

destroy, 

NULL, 

NULL); 


box 

// non-modal window 

// position is determined by Windows 
// object to show 
// number of columns 
// widths of columns 

// function that returns number of lines 
// function that generates a line 
// window title 

// use the default icon for the window 
// position the cursor on the first line 
// "kill" callback 
// "new" callback 
// "update" callback 
// "edit" callback 

// function to call when the user pressed Enter 
// function to call when the window is closed 
// use default popup menu items 
// use the same icon for all line 


The find memcpy plug-in has many of the callbacks set to NULL. However most of the 
callbacks are not needed. It is not common to add new lines to a list box. The popup menu 
callback can be useful for operating on list data in ways other than jumping to the disassembly 
for a single item. 

The key callbacks are size, description, enter, and destroy. 
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The size callback returns the number of lines to display in the list box. There is not 
much to it unless items are being added or removed from the list. The prototype for find 
memcpy’s size function is: 

ulong idaapi size(void* obj) 

The description callback fills in the rows for the list box. It is called for every item in the 
list. The function is passed the object, line number, and arrptr. The last item is an array of 
pointers for column data. The description function copies the text data it wishes to display 
into the array. Description setups the column header when passed 0 for the n argument. 

The following code from find memcpy copies the headers column headers into arrptr. 

static const char* header[] = {"Address", "Type", "Movsd/b distance"}; 
void idaapi description(void *obj,ulong n,char * const ‘arrptr) 

{ 

if ( n == 0 ) // sets up headers 
{ 

for ( int i=0; i < qnumber(header) ; i++ ) 
qstrncpy(arrptr[i] , header[i], MAXSTR); 
return; 

} 

The enter callback is generally used to jump to an address. The function is called when 
the user presses Enter, or double clicks on a line in the chooser list. The function is passed 
the object and line number. 

void idaapi enter(void * obj, ulong n) 

The destroy callback is called when the chooser list is being destroyed. Destroy can 
perform resource cleanup as is the case in find memcpy. 

void idaapi destroy(void* obj) 

{ 

netnode ‘node = (netnode *)obj; 

node->kill(); 

return; 

} 

Conclusion 

The find memcpy plug-in is an introduction to the IDA API. The next plug-in uses and 
builds upon many of the same functions presented in this section. 
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The Indirect Call Plug-in 

The IDC section presented a script to find and create cross references for indirect calls 
through aVTable.This solution requires knowing where interestingVTables are located. 
Instead of observing the targets of aVTable, the opposite approach can be taken by seeking 
out all the callers. Callers would include any indirect jump instruction. However, for the sake 
of brevity indirect calls will refer to both indirect calls and jumps. 

Proposed Strategy 

1. Similar to the find memcpy plug-in, the binary is scanned for interesting instructions, 
in this case, indirect calls. 

2. Breakpoints are set on all indirect calls. 

3. The plug-in adds a callback to the debugger. 

4. The debugger instruments the binary. 

5. The callback records information. Optionally the callback performs a step into the 
call target and record the address. 

6. Breakpoints are removed when the process exits. 

7. Data is presented to the user and optionally cross references are added. 

Based on the proposed strategy four separate tasks need to be performed. Separating the 
tasks allows code to be written for parts that can be replaced at a later point. Fully working 
chooser lists are not needed immediately, during development writing to the message window 
will suffice. 

■ Collect data 

■ Query user for options 

■ Implement the callback 

■ Present results to the user 

The plug-in is presented later in the chapter. However, relevant code and screenshots 
will be shown in the following sections. 

Collecting Data 

Before starting to collect data, we need data structures to store them in. During the writing of 
the plug-in the data container changed but netnodes remained the main data structures. The 
plug-in uses two netnodes and a qvector. Netnodes were introduced in the previous plug-in. 
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Qvector is also an SDK data type and is defined in pro.h. It supports most of the standard 
vector methods. 


Name 

DataType 

Description 

Internal Type 

calls 

Netnode 

Contains all found indirect calls 

indirect_t 

vtables 

Netnode 

Contains all found VTables 

vtablejt 

bplist 

qvector 

Index list into calls netnode 

ulong 


The netnodes use altval and supval arrays to allow both address lookup as well as 
iteration of objects. The altval sparse array is accessed by address. The value contained in 
the atval is an index into the supval array. Altval arrays are initialized to 0. Thus supval 
indexing begins at 1. The following is some example code taken from the indirectCalls 
header comments. 

// .text:030CC0FB call dword ptr [eax+3Ch] ; 
indirect_t myObj; 

ulong index = calls->altval(0x030CC0FB); 

if (index != 0) // indirect call (assume we assigned it earlier) 

{ 

indirect_t myObj = calls->supval(index, SmyObj, sizeof(myObj)); 
msg("%a -> %a\n", myObj.caller, myObj.target) ; 

} 

Collection of data is performed by the findlndirectCalls function. The current segment is 
scanned, not only functions.The function should collect defined indirect calls although not 
within a proper function. The following code scans for the calls. 

switch (cmd.itype) 

{ 

case NN_callfi: 
case NN_callni: 
case NN_jmpfi: 
case NN_jmpni: 


if (get_first_fcref_from(cmd.ea) == BADADDR && 

get_first_dref_from(cmd.ea) == BADADDR) //no fwd xref 
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{ 

indirect_t currcall; 
filllndirectObj(currcall) ; 
if (cmd.itype & NNJMPxI) // jmp? 

{ 

currcall. flags |= JMPSETFLAG; 

} 

node->altset(cmd.ea, counter); // altval keyed by addr 
node->supset(counter++, Scurrcall, sizeof(currcall)); 

The code is similar to the previous plug-in as it analyzes the instruction and checks for 
certain nmemonics. Cross references checks from the call are performed for both code and data. 
The data cross reference check is necessary to avoid jump tables. The call to filllndirectObj 
prepares the indirectjt object. Some more instruction decoding takes place which is recorded 
into the object. 

If the call is an indirect near call, further processing takes place. The goal is to determine 
if the call is of the form: 

call [reg] or call [reg + offset] 

The preceding calls particularly with an offset may contain a VTable address in the 
register. The register is extracted and stored in the object along with any offset. Note that 
the information is located within the operands type attribute.With the register and offset, 
the target address can be calculated. In theory all call instructions could be decoded. Finally 
there is a test checking a flag to determine if a call is a jmp. This is done in order to set the 
appropriate cross reference type if the call is completed during runtime. 

This function concludes the collection of information prior to acquiring options from 
the user. 

User Interface 

Various options are available to the user. AskUsingForm_c API call creates the user interface. 
(See Figure 9.18) 

Certain characters control whether a checkbox or radio button appears. There is not 
much documentation available: however there is some sample code. (http://www.openrce. 
org/ downloads/details/32/User_Interface_Sample_Code) 
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Figure 9.18 Indirect Call User Interface 


Indirect Call Plugin 


WARNING: Plugin executes the binary undei the debugger. 
Ensure the process options have been set. 


Found Qx55C indirect calls without xrefs 

(♦) Run Debugger 
O Only collect information 

0 Display indirect call 1st 
0 Display BPs hit 
0 Display cross segment BPs hit 
0 akc the weife j 

[ I Make the xrefs for cross segment calls 
0 Display possible vlables 
0 Include non-offset[call [eax|) calls for vtables 


Ok 


Cancel 


Help 


The options are processed and the if the user chooses to run the debugger a new API 
call is made to hook notification of the debugger. 

if (!hook_to_notification_point(HT_DBG, callback, &gDbgOptions)) 

{ 

warning("Could not hook to notification point\n"); 

register_event(E_HOOKFAIL); 

return; 

} 

The following is the function retype as well as supported hook types. 

HT_IDP, // Hook to the processor module. 

HT_UI, // Hook to the user interface. 

HT_DBG, // Hook to the debugger. 

HT_IDB, // Hook to the database events, 

idaman bool ida_export hook_to_notification_point( 

hook_type_t hook_type, 
hook_cb_t *cb, 
void ‘user data); 
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The first argument is the type of hook. The second argument is the callback function 
that receives notification. The final argument can be NULL. Passing an object serves two 
purposes. In order to unhook the same object must be used. The second purpose is passing 
data to the callback function. In this case the passed user_data is a global. 

Finally breakpoints are set and the process is started using the start_process call. 

int idaapi start_process(const char *path, const char *args, 

const char *sdir) 

If the arguments are NULL , start_process uses data previously entered under Debugger | 
Process options. The callback is set and the debugger should be running. 

Implementing the Callback 

The debugger starts and the callback patiently waits for events. The following is the callback’s 
prototype. 

int idaapi callback(void* user_data,int notification_code,va_list va) 

The notification code describes the type of event being received. There are various types 
of notification from low level ones dealing with library loading to higher level breakpoint 
notifications. The notifications are documented in the dbg_notijication_t enum located in 
dbg.hpp. The callback has a switch and handles three types of notification. 

dbg_bpt 

dbg_bpt is the breakpoint notification. The portion of code that handles dbg_bpt has three 
possible outcomes. 

■ The breakpoint address is not the calls netnode.This is a user set breakpoint and 
should be handled as such. The plug-in calls suspend_process and exits the callback. 
suspend_process(); 
return 0; 


■ The breakpoint was set by the plug-in however the call instruction is not one of 
the predecoded types. The caller address is stored is Iast_bp, since the target won’t be 
resolved until the step_into.A call is made to request_del_bp and reqliest_step_into.T he 
step_into function cannot be called from a notification handler. 

last_bp = from; // saves the caller address 

request_del_bpt(from); // queue request_del_bpt() 

// 

// From: dbg.hpp request_step_into() AND step into() 
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// Type: Asynchronous function - available as Request 
// In Notification handler it is MANDATORY to call 
// Async function in request form 
request_step_into() ; // queue a request_step_into() 

// request will be run after all notification handlers 
run_requests(); 
break; 

■ The breakpoint was set by the plug-in and the calling instruction is one of the 
predecoded types. The my_indirect object contains both the register number and 
offset (could be zero). The register is read using get_reg_val. 

The register value is then stored into vtaddr. This is assumed to be a VTable 
address. In order to recover the target address a VTable lookup needs to be 
performed. However, reading memory while in a notification handler can provide 
unreliable results. The database and process may not be in sync. The issue was 
observed during the development of this plug-in. The invalidate_dbgmem_contents 
function invalidates and flushes IDA’s cache. 

Inside a notification handler calling invalidate_dbgmem_contents is required before 
reading and writing memory. Another option is invalidate_dbgmem_config which 
although slower is more thorough. Both are defined in bytes.hpp. 

Two more functions are called, add VTable and setTargetXref. Assuming user 
options permit, the functions will a create cross reference and possibly a VTable. 

// copy register_t struct in regval 
get_reg_val(regname[my_indirect.call_reg] , &regval); 

// vtaddr == VTable base addr 
vtaddr = (ea_t)regval.ival; 

// flushes IDA's cache 

invalidate_dbgmem_contents((ea_t)regval.ival, 0x100 + 

my_indirect.offset); 

// read target address from table 
to = get_long(my_indirect.offset + vtaddr); 
my_indirect.target = to; 

// 

addVTable(my_dbg,vtaddr, &my_indirect); 

setTargetXref(my_dbg, index, &my_indirect); 

calls->supset(index, &my_indirect, sizeof(my_indirect)); 

del_bpt(from); 

continue_process(); 

break; 
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dbg_step_into 

■ dbg_step_into is the step_into notification. The notification is caused either by the 
user or the request_step_into call. If the user caused the notification, suspend_process 
is called. 

The current EIP is the target of the call. The address is copied into the object. 
setTargetXref adds a cross reference based on user options. 

from = last_bp; 
if (from == BADADDR) 

{ 

suspend_process() ; // user caused step_into 
return 0; 

} 

long index = calls->altval (from); // index into supval 
get_reg_val("EIP", Sregval); // current EIP is the 'to' 

to = (ea_t)regval.ival; 
indirect_t my_indirect; 

calls->supval(index, &my_indirect, sizeof(my_indirect)); 
my_indirect.target = to; 

// Add cross reference based on user options and checks 
setTargetXref(my_dbg, index, &my_indirect); 

// save completed indirect_t object 
calls->supset(index, &my_indirect, sizeof(my_indirect)); 

// reset last_bp and continue the debugger 
last_bp = BADADDR; 
continue_process() ; 
break; 

db g_p ro c ess_exit 

This notification signals the termination of the debugged process. 

unhook_from_notification_point(HT_DBG, callback, user_data); 
requestDelBps(calls) ; 
run_requests() ; 
register_event(E_PROCEXIT); 

if (options & DISPLAY_INCALLS) 

createlndirectCallWindow(calls); 

if (options & DISPLAY_BPS) 

createCompletedBpWindow(calls, my_dbg->bplist); 


www.syngress.com 


IDA Scripting and Plug-ins • Chapter 9 263 


if (options & DISPLAY_VTABLES) 

createVTableWindow(my_dbg->vtables) ; 

Presenting Results 

The V Table display includes an estimated VTable size. The size is estimated by iterating through 
pointers and checking for references. The rest of the presentation functions are similar to the 
find memcpy plug-in. There are two new function introduced in the description callbacks. They 
both deal with presenting text. The first is get_nice_colored_name. This function can construct 
addresses as seen listed in IDA, such as segmentiaddress. Various flags specify the format. 


# define 

GNCN 

NOSEG 

0x0001 

// 

ignore the segment prefix 






//producing the name 


# define 

GNCN 

NOCOLOR 

0x0002 

// 

generate an uncolored name 

# define 

GNCN 

NOLABEL 

0x0004 

// 

don't generate labels 


# define 

GNCN 

NOFUNC 

0x0008 

// 

don't generate funcnamet... 

expressions 

# define 

GNCN 

SEG FUNC 

0x0010 

// 

generate both segment and 

function names 

(default is to 

omit segment 

name if a function name is present) 

# define 

GNCN 

SEGNUM 

0x0020 

// 

segment part is displayed 

as aa hex numbe: 

# define 

GNCN 

REQFUNC 

0x0040 

// 

return 0 if the address does not 





// 

belong to a function 


# define 

GNCN 

REQNAME 

0x0080 

// 

return 0 if the address can only be 


// represented as a hex number 


// returns: the length of the generated name in bytes 
// The resulting name will have color escape characters 
// GETN_NOCOLOR was not specified 
// (see lines.hpp for color definitions) 

idaman ssize_t ida_export get_nice_colored_name( 
ea_t ea, 
char *buf, 
size_t bufsize, 
int flags=0) ; 

The second new function demangles names. By default IDA uses mangled names, although 
the option can be changed. This following function produced a short demangled name. 

inline char *get_short_name(ea_t from, ea_t ea, char *buf, size_t bufsize) 

Both of the functions are located in names.hpp. The plug-in was run against jscipt.dll from 
IE7. Figure 9.19 is the list of all indirect calls. 
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Figure 9.19 Indirect Call List from jscript.dll 
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Figure 9.20 Completed Call List from jscript.dll 
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Address Function X. Instruction 

text:6333DB63 DIIGetClas$Obfect(x,x.x) x call dword ptr [e 

text:6338D881 DIIGetClas$Object(x,x,x) x call dword ptr (e 

text:G338DC7E CCIassFactory: Querylnterf.. x call dword ptr (e 

. text:6333D B 83 DIIGetClassObject(xxx) x call dword ptr [e 

. text:6333E357 CJScriptClassFactory::Cre.. x call dwordptr[e 

. text:6333E 37A GJScriptClassFactory::Cre.. x call dwordptr[e 

text'633A0FCE COIeScript :Querylntertace. x call dword ptr [e 

text:6338E381 CJ$criptClas$Factory::Cre. x call dword ptr (e 

text:6338D3A5 CSe$$ion;:AddObiec»(u$lio,. x call dword ptr 

text:6339BD6A NameTbl.:CreateVar(SVM ... x call dwordptrfe 

text:633A213F NarneTbl::GetVal(SYM MA. x call dword ptr [e 

text:633A2153 NameTbl::GetVal(SYM *.V... x call dwordptr[e 

. text:6339203D CO leS cript :GetScriptDisp. x call dword ptr [e 

. text:6338B 625 NameTbl :Querylntertace(., x call dwordptr[e 

text:63338479 NameTbl: GetDi$plD(usho x call dword ptr (e 

.text:6339CC03 NameTbt:GetVval|SYM x call dword ptr [e 

text:63395F23 CScriptBody: CScriptBodyl... x call dword ptr [e 

texl:6339D5D5 CScriptBody:.Relea$e(void) x call dwordptr[e 

text:6339D602 CScnptBody: Retease(void) x call dwordptr[e 

text6338CD26 CSession:GetTypeProto(i. x call dword ptr (e 

text:633A31C3 CSessiori::Execute(Name. x call dwordptr[e 

text:633A35AE CScriptRuntime::lnit(CSess... x call dword ptr (e 

.texl:633A35E3 CScnptRuntime::lnit(CSe$s.. x call dword ptr [e 

text:633A35EB CScriptR untime:: I mt(CS ess., x call dwoidptr[e 

text:633A38A4 CScnptRuntime::lmtThis(V... x call dword ptr [e 

,text:633A3822 TLS_NoDestructor::TLS_,.. x call dword ptr [e 

Line 122 of 123 

633a8eb8 new vtable caller; 6339bei3 , to: 633a2b3l~ 

6339bf38 NEW VTABLE Caller: 6339C21d , to: 633a29Cl 

63391090 NEW VTABLE Caller: 6338Cf3b , to: 63391171 

633a0a70 new vtable caller: 633911f7 , to: 6339a893 

633a02f8 NEW VTABLE caller: 633Scee2 . to: 633a29cl 

63390f28 NEW VTABLE caller: 6338cef9 , to: 6339a893 

63392b28 NEW VTABLE caller: 6339C67f , to: 6339Clbf 

633a9dd8 NEW vtable caller: 633aa6d9 , to: 6339a893 

Debugger: Process terminated by the user, 
requestoelsps size: S5d 


call dword ptr [eax+14h) 
call dword ptr (eax) 
call dword ptr (ecx+4) 
call dword ptr [eax+8] 
call dword ptr [eax+14h] 
call dword ptr [eax] 
call dword ptr (ecx+4) 
call dword ptr (eax+8) 
call dword ptr |ebx+3Ch] 
call dword ptr [eax+QA4h] 
call dword ptr [eax+QACh] 
call dword ptr [eax+OBCh] 
call dword ptr (ecx+4) 
call dword ptr (eax+4) 
call dword ptr (eax+084h) 
call dword ptr [eax+0B8h) 
call dword ptr (ecx+4) 
call dword ptr [eax+4] 
call dword ptr [eax+8) 
call dword ptr (ecx+8) 
call dword ptr [eax+30h] 
call dword ptr (ecx+4) 
call dword ptr (ecx+4) 
call dwoid ptr [eax+ODOh] 
call dword ptr [eax+74h] 
call dword ptr (ecx+4) 


Target 

text:6338E3F9 

text:6338DC3E 

texl:6338D2F5 

.text:6338DAA1 

text:6338E3F9 

.text:633A0F2D 

.text 633A0D85 

text:633A0CF1 

.text:6339BD59 

text:6339A893 

.text:633A21 G9 

text:633A1DC3 

.text:633A2831 

text:633A2831 

text:63392053 

.text.6338C361 

text:633AO085 

text:633A0D85 

text:633AQCF1 

.text'633A2SCl 

text:633A2F46 

text:633A0D85 

- text:633A2B 31 

text:633A346B 

text633A388C 

text:633AQ085 


_ 3tajl 

_ a * 

- ♦ y H e a cd i P »| 

~S)aar I a 

ffi -'i U T,|| A & V A A 


T arget Function 

CJ S criptClassFactory:: G etClassI D (void) 
CCIas$Factory:;QuerylnterfaceLGUIO co. . 
AutB lock:: AddR ef (void) 

CCIassFactory:: R elea$e( void) 

CJS ciiptClassFactory.: G etClassI D (void) 
COIeSaipt: QuerylnterfaceLGUID const... 
COIeScript AddRef(void) 

CO leS cript: R elease(void) 

NameT bl::CreateVar(SYM '.ulong.VAR M... 
NameTbl. CreateVvalfSYM \WAL* Mon. 
ModuleBinder:.GetVval(SYM X .WAL x Mo... 
NameTbL:GetValCore(WAL *,VAR 1 
N ame T bt: AddR ef(void) 

N ameT bl:: AddR ef (void) 
Module6mder::GetVva©i$p(SYM "WAL 
GlobalBinder :EnsuieBuiltin(SYM “,WAL"... 
COIeScript AddRef(void) 

COIeScript: :AddRef(void) 

CO leS cript: R elease( void) 

N ame T bl: R eleasel void) 

ScrfncO bj Call(VAR MrttVAR ".VAR 
COIeScript :AddRef(vcndl 
NameT bl.;AddRef(void) 

S crFncO bj. G etl nfo(void) 

ModuleBmdec: PdispThis(void) 

CO leS cript: AddR ef (vord) 


Down Disk: “1G8 
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Figure 9.21 Viable List from jscript.dll 
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.text:633A08D0 

0008 
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N ameT bl:: R elease(void) 

OOCC 

0033 

. text:633A42B 8 
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D exCaller:: R eleasef void) 
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0004 
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0008 

text:6339ED29 

G cAlloc:: R eleasef void) 
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.text:633A8EB8 

0008 

text:633A29C1 

N ameT bl:: R eleasef void) 

00DC 

0037 

,text:6339BF38 

00A4 

text:6339C3D3 

ArrayObj::CreateVval(SYM X ,WAL x Mong x ) 

OOCC 

0033 

text:63391090 

00D8 

text:63391171 

S tringFncO bj:: CreateTypeProxyfN ameT bl. 

00DC 

0037 

text:633AQA70 

00A8 

text:6338CE39 

N ameT bl:: CreateProtoR efVvalfSYM X ,W. . 

OOCC 

0033 

text:633A02F8 

0008 

text:633A29C1 

N ameT bl:: R eleasef void) 

00DC 

0037 

.text:63390F28 

00A4 

text:6339A893 

NameTbl::CreateVval(SYM X ,WAL x Mon... 

OOCC 

0033 

text:63392B28 

0078 

text:6339C1BF 

ArrayFncObj::Construct(VAR Mnt,VAR x ) 

00DC 

0037 

,text:633A9DD8 

0QA4 

text:6339A893 

N ameT bl:: CreateVvalfS YM X ,WAL x Mon... 

00DC 
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633a8eb8 

NEW 

VTABLE 

cal 1 er: 

63 3 9bel3 

, to: 

63 3 a2 b31 

63 3 9bf3 8 

NEW 

VTABLE 

cal 1 er: 

63 3 9C2Id 

, to: 

63 3 a2 9Cl 

63391090 

NEW 

VTABLE 

cal 1 er: 

6338cf3b 

, to: 

63391171 

633aoa70 

NEW 

VTABLE 

cal 1 er: 

63 3 911f7 

, to: 

63 3 9a893 

63 3 d02 f8 

NEW 

VTABLE 

cal 1 er: 

6338cee2 

, to: 

63 3 a2 9Cl 

63 3 90f2 8 

NEW 

VTABLE 

cal 1 er: 

63 3 8cef9 

, to: 

63 3 9a893 

63 3 92 b2 8 

NEW 

VTABLE 

cal 1 er: 

63 3 9c67f 

, to: 

63 3 9clbf 

633a9dd8 

NEW 

VTABLE 

cal 1 er: 

633aa6d9 

, to: 

63 3 9a893 


Debugger: Process terminated by the user. 

requeitDelBps si2e: 55d 


AU: idle Down Disk: 4GB 


Figure 9.22 Indirect Call Plug-in indirectCall.h 

* Indirect Call IDA Pro plugin 

•k 

* Copyright (c) 2008 Luis Miras 

* Licensed under the BSD License 

* 
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#ifndef INDIRECTCALLS_H_ 

#define INDIRECTCALLS_H_ 

#define NODE_COUNT -1 
#define NNJMPxI 0x4 0 

#define CNAMEOPT (GNCN_NOCOLOR | GNCN_NOFUNC | GNCN_NOLABEL) 

struct dbgOptions; //fwd declaration 
struct indirectCallObj; //fwd declaration 
typedef indirectCallObj indirect_t; 
typedef qvector<ulong> bphitlist_t; 
long vtEstimateSize(ea_t); 

void idaapi vtDescription(void *,ulong, char * const *); 

void idaapi vtEnter(void * ,ulong); 

void idaapi vtDestroy(void*) ; 

void createVTableWindow(netnode* vtables); 

void idaapi icDescription(void *,ulong ,char * const *); 

void idaapi icEnter(void * ,ulong); 

void idaapi icDestroy(void*) ; 

ulong idaapi size(void*); 

void createlndirectCallWindow(netnode*) ; 

void idaapi ccDescription(void *,ulong ,char * const *); 
void idaapi ccEnter(void* ,ulong) ; 
void idaapi ccDestroy(void*) ; 
ulong idaapi ccSize(void*) ; 

void createCompletedBpWindow(netnode* , bphitlist_t*); 

void requestSetBps(netnode*) ; 

void setBps(netnode*) ; 

void requestDelBps(netnode*) ; 

void delBps(netnode*) ; 

void setTargetXref(dbgOptions* , long , indirect_t*) ; 

void addVTable(dbgOptions* , ea_t , indirect_t*); 

int idaapi callback(void* , int , va_list); 

void filllndirectObj(indirect_t &); 

bool setnodesize(netnode* , long); 

long getnodesize(netnode*) ; 

long getobjcount(netnode*) ; 

void findlndirectCalls(segment_t* , netnode*); 
void closeListWindows(void) ; 
void register_event(ulong) ; 
void run(int); 
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int init(void) ; 
void term(void); 
struct indirectCallObj 


ea t caller; 

// 

indirect caller 

address 

ea t target; 

// 

target address 


ea t offset; 

// 

valid for call 

[reg+offset] 


// 

defaults to 0 


short call reg; 

// 

enum REG 


short flags; 

// 

enum callflags t 



In¬ 
struct vtableObj 
{ 

ea_t baseaddr; // baseaddr reg in call [reg + off] 

ea_t largestOffset; // largest off seen in call [reg + off] 

} ; 

typedef struct vtableObj vtable_t; 
typedef qvector<ulong> bphitlist_t; 
struct dbgOptions 
{ 

netnode* calls; 
netnode* vtables; 
bphitlist_t* bplist; 
ulong options; 

In¬ 
struct completedbp 


netnode* calls; 
bphitlist_t* callindex; 

} ; 

typedef completedbp completedbp_t; 


enum uioptions_t { 
DISPLAY_INCALLS = 
DIS PLAY_B PS 
DISPLAY_XS_BPS 
MAKE_XREFS 
MAKE_XS_XREFS 
DISPLAY_VTABLES = 
INC NONOFF CALLS = 


0x0001, 

0x0002, 

0x0004, 

0x0008, 

0x0010, 

0x0020, 

0x0040 
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} ; 

enum callflags_t { 

JMPSETFLAG = 1, 

XRSETFLAG = 2, 

XSEGFLAG = 4 

} ; 

char* regname[] = {"EAX","ECX","EDX","EBX","ESP","EBP","ESI","EDI"}; 
enum REG {eax, ecx, edx, ebx, esp, ebp, esi, edi, none = -1}; 
enum EVENTS 
{ 

E_START, E_CANCEL, E_OPTIONS, E_HOOKFAIL, E_PROCFAIL, 

E_DWCALL, E_DWXREFS, E_DWVTABLE, E_PROCEXIT 

} ; 

// incomplete calls, choose2() list box 
char icTitle[] = "Indirect calls" ; 

static const char* icHeader[] = {"Address", "Xref","Function", "Instruction"}; 
static const int icWidths[] = {16, 4, 36, 20}; 

// completed calls, choose2() list box 
char ccTitlef] = "Completed calls" ; 

static const char* ccHeader[] = {"Address", "Function", "Xref", "Instruction", 
"Xseg","Target", "Target Function"}; 

static const int ccWidths[] = { 16, 28, 4, 18, 4, 16, 28}; 

// vtables, choose2() list box 
char vtTitlef] = "VTables"; 

static const char* vtHeader[] = {"VTable ", "Largest offset seen", "Offset 
target", "Offset function", "Estimated size", "Estimated function count"}; 
static const int vtwidths[] = { 16, 16, 16, 28, 16, 20}; 

// ui string AskUsingForm_c() 
const char preformat[] = 

"STARTITEM 0\n" 

// Help 
"HELP\n" 

"This plugin searches for indirect calls. For example:\n" 

"\n" 

"call dword ptr [eax+14h]\n" 

"jmp eax\n" 

"\n" 

"Breakpoints are set on all the calls.\n" 
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"A breakpoint handler will:\n" 

" 1. Determine if one of its breakpoints triggered.\n" 

" 2. Delete the breakpoint\n" 

" 3. Step into the call\n" 

" 4. Record both the caller and callee addresses\n" 

"\n" 

"ENDHELP\n" 

// Title 

"Indirect Call Plugin\n" 

// Dialog Text 

"WARNING: Plugin executes the binary under the debugger.\n" 

"Ensure the process options have been set.\n\n" 

"Found Ox%a indirect calls without xrefs\n\n" 

// Radio Buttons 
"<#Runs the debugger#" 

"Run Debugger:R>\n" 

"<#Collects data on indirect calls#" 

"Only collect information:R>>\n" 

// Check Boxes 

"<# Create indirect call window. #" 

"Display indirect call list :C>\n" 

"<# Create BP window. #" 

"Display BPs hit :C>\n" 

"<# Include cross segment BPs in BP window. #" 

"Display cross segment BPs hit :C>\n" 

"<# Automatically create xrefs btwn caller and target. #" 

"Make the xrefs :C>\n" 

"<# Automatically create xrefs btwn caller and target in different segments. # 
"Make the xrefs for cross segment calls:C>\n" 

"<# Create a vtable window #" 

"Display possible vtables :C>\n\n" 

"<# May lead to false positives (not recommended) #" 

"Include non-offset(call [eax]) calls for vtables :C>>\n\n"; 


#endif /* INDIRECTCALLS H */ 
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Figure 9.23 Indirect Call Plugin indirectCall.cpp 

* Indirect Call IDA Pro plugin 


* Copyright (c) 2008 Luis Miras 

* Licensed under the BSD License 

* 


* Requirements: 

★ 

~k 

~k 

~k 

* Description: 

* 

~k 

•k 

★ 


This plugin works alongside the IDA Pro debugger. 

The plugin requires x86 processor. The plugin "should" 
work under the IDA Linux debugger. It has not been 
tested. 

The plugin attempt to create cross references for 
indirect calls/jmps. For brevity indirect calls/jmp 
will be refered only as indirect calls. The plugin 
also attempts to identify vtables. 


* Strategy: The binary's current segment is scanned for indirect 

* calls. The binary is instrumented under the debugger. 

* A breakpoint handler either calculates the target or 

* steps into the target. Depending on user options cross 

* references will be made and possible vtables listed. 

k 


* Data structures: netnode and qvector are used. Both are built in IDA 

* types, minimizing 3rd party dependencies, netnodes 

* allow for persistent data,they are saved in the IDB 

* However, in this plugin the netnodes are kill()'ed 


* netnodes are implemented internally as B-trees. 

* IDA uses netnodes extensively for its own storage. 

* netnodes are defined in netnode.hpp. 

* 

* netnodes in the plugin: calls - holds all indirect calls 

* vtable - holds all vtables 

•k 

* netnodes have various internal data structures. 

* The plugin uses 2 types of arrays: 

* altval - a sparce array of 32 bit values, initially set to 0. 

* supval - an array of variable sized objects (MAXSPECSIZE) 
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~k 

* Addresses are used as keys into altval array. The value at the key 

* is then used as an index into the supval array. The supval array 

* holds an object of variable size. 

* 

* This allows fast lookup using address keys, while being able to 

* iterate through all items using supval. 

~k 

* An example: 

* 

* .text:030CC0FB call dword ptr [eax+3Ch] 

* indirect_t myObj; 

* ulong index = calls->altval(0x030CC0FB); 

* 

* if (index != 0) // indirect call (assume we assigned it earlier) 

* { 

* indirect_t myObj = calls->supval(index, SmyObj, sizeof(myObj)); 

* msg("%a -> %a\n", myObj.caller, myObj.target); 

* } 

* 

* the calls netnode holds indirect_t objects 

* the vtables netnode holds vtable_t objects 

* bphitlist_t is a qvector that holds indexes into the calls netnode 

★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★it **j 

#include <ida.hpp> 

#include <idp.hpp> 

#include <dbg.hpp> 

#include <loader.hpp> 

#include <allins.hpp> 

#include <intel.hpp> 

#include "indirectCalls.h" 

dbgOptions gDbgOptions = {NULL, NULL, NULL, 0); 

/************************************************************************** 

* Function: vtEstimateSize 

* Args: ea_t addr - base address of a VTable 

* Return: long - Estimated VTable length 
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* This function attempts to calculate the size of a vtable given its 

* base address. It checks xrefs to determine if still in a vtable 

* 

.text:03010D34 off_3010D34 
.text:03010D34 
.text:03010D38 
.text:03010D3C 
.text:03010D4 0 
.text:03010D44 
.text:03010D4 8 

.text:03010D54 off_3010D54 

•k 

* Sometimes a string is stored at the end of a vtable as in this case. 

* vtEstimateSize doesn't understand anything other than dword ptrs 

kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k/ 

long vtEstimateSize(ea_t addr) 

{ 

flags_t flags; 
ea_t curraddr = addr; 
ea_t lastaddr = addr; 
bool done = false; 

curraddr = next_head(lastaddr, BADADDR); 
while (!done) 

{ 

if (curraddr - lastaddr != 4) // DWORD size differences 

{ 

done = true; 

} 

flags = getFlags (curraddr) ; 
if (! done && ! isDwrd (flags) ) 

{ 

done = true; 

} 

// a dref_to could suggest the start of a new vtable 
if (!done && get_first_dref_to(curraddr) != BADADDR) 
done = true; 
if (!done) 


dd offset sub_308A561 

dd offset sub_3082D8D 
dd offset sub_3082DA6 
dd offset sub_3091542 
dd offset sub_30B9110 
dd 75667608h, 6174636Eh, 62h ; 

; 8 'vfunctab' 

dd offset sub 308A561 
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{ 

lastaddr = curraddr; 

curraddr = next_head(lastaddr, BADADDR); 

} 

} 

return lastaddr - addr + 4; 

} 

/************************************************************************** 

* Function: vtDescription 

* 

* This is a standard callback in the choose2() SDK call. This function 

* fills in all column content for a specific line. Headers names are 

* set during the first call to this function, when n == 0. 

* arrptr is a char* array to the column content for a line. 

* arrptr[number of columns] 

* 

* vtDescription creates 6 columns based on the vtHeader array 

************************************************************************** I 

void idaapi vtDescription(void *obj,ulong n,char * const *arrptr) 

{ 

netnode *node = (netnode *)obj; 
vtable_t curr_vtable; 
ea_t target; 
long vtSize; 

if ( n == 0 ) // sets up headers 

{ 

for ( int i=0; i < qnumber(vtHeader) ; i++ ) 
qstrncpy(arrptr[i], vtHeader[i], MAXSTR); 
return; 

} 

// Empty netnode 
if (!getobjcount(node)) 
return; 

char buffer[MAXSTR]; 

node->supval(n, &curr_vtable, sizeof(curr_vtable)); 
vtSize = vtEstimateSize(curr_vtable.baseaddr) ; 

target = get_long(curr_vtable.largestOffset t curr_vtable.baseaddr); 

get_nice_colored_name(curr_vtable.baseaddr, 

arrptr[0], MAXSTR, CNAMEOPT); 
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qsnprintf(arrptr[1], MAXSTR, "%04a", curr_vtable.largestOffset); 
get_nice_colored_name(target, arrptr[2], MAXSTR, CNAMEOPT); 

get_short_name(BADADDR, target , buffer, MAXSTR); //demangles fname 


qsnprintf(arrptr 

[3] , 

. MAXSTR, 

"%s". 

buffer); 

qsnprintf(arrptr 

[4] , 

, MAXSTR, 

"%04a' 

', vtSize) ; 

qsnprintf(arrptr 

[5] , 

. MAXSTR, 

" % 0 4 a' 

', vtSize/4) 


return; 

} 

/************************************************************************** 

* Function: vtEnter 

★ 

* This is a standard callback in the choose2() SDK call. This function 

* is called when the user pressed Enter or Double-Clicks on a line in 

* the chooser list. 

**************************************************************************j 

void idaapi vtEnter(void * obj,ulong n) 

{ 

vtable_t curr_vtable; 

netnode *node = (netnode *)obj; 

node->supval(n, &curr_vtable, sizeof(curr_vtable)); 

jumpto(curr_vtable.baseaddr) ; 

return; 

} 

j************************************************************************** 

* Function: vtDestroy 

* 

* This is a standard callback in the choose2() SDK call. This function 

* is called when the chooser list is being destroyed. Resource cleanup 

* is common in this function. In this case any resource 

* cleanup is handled by register_event() . 

**************************************************************************j 

void idaapi vtDestroy(void* obj) 

{ 

netnode *node = (netnode *)obj; 
msg("\"%s\" window closed\n", vtTitle); 
register_event(E_DWVTABLE) ; 
return; 

} 
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* Function: createVTableWindow 

* 

* A wrapper around choose2() API. 'Generic list chooser (n-column)' 

* This sets up the callbacks and necessary options. 

* NOTE: 1. Cannot free the "object to show" until chooser closes 

* 2. Cannot unload plugin until chooser closes, 

* removing callbacks. 

**************************************************************************/ 
void createVTableWindow(netnode* vtables) 


choose2(false, 

- 1 , - 1 , - 1 , - 1 , 
vtables, 

qnumber(vtHeader), 

vtwidths, 

size, 

vtDescription, 
vtTitle, 

- 1 , 

0 , 

NULL, 

NULL, 

NULL, 

NULL, 
vtEnter, 
vtDestroy, 

NULL, 

NULL); 


// non-modal window 

// position is determined by Windows 
// object to show 
// number of columns 
// widths of columns 

// function that returns number of lines 
// function that generates a line 
// window title 

// use the default icon for the window 
// position the cursor on the first line 
// "kill" callback 
// "new" callback 
// "update" callback 
// "edit" callback 

// function to call when the user pressed Enter 
// function to call when the window is closed 
// use default popup menu items 
// use the same icon for all line 


} 

^'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k 


* Function: icDescription 

•k 


* This is a standard callback in the choose2() SDK call. This function 

* fills in all column content for a specific line. Headers names are 

* set during the first call to this function, when n == 0. 

* arrptr is a char* array to the column content for a line. 

* arrptr[number of columns] 

* 


* vtDescription creates 4 columns based on the icHeader array 
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-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k^ 

void idaapi icDescription(void *obj,ulong n,char * const *arrptr) 

{ 

netnode *node = (netnode *)obj; 
indirect_t curr_indirect; 

if ( n == 0 ) // sets up headers 

{ 

for ( int i=0; i < qnumber(icHeader) ; i++ ) 
qstrncpy(arrptr[i], icHeader[i], MAXSTR); 
return; 

} 

// list empty? 
if (!getobjcount(node)) 
return; 

char buffer[MAXSTR]; 

node->supval(n, &curr_indirect, sizeof(curr_indirect)); 
func_t* currFunc = get_func(curr_indirect.caller) ; 

ua_anaO(curr_indirect.caller) ; 

get_nice_colored_name(curr_indirect.caller, 

arrptr[0], MAXSTR, CNAMEOPT); // address 

if (curr_indirect .flags & XRSETFLAG) 
qstrncpy(arrptr[1], "x", MAXSTR); 

else 

qstrncpy(arrptr[1], MAXSTR); 

get_short_name(BADADDR, currFunc->startEA, buffer, MAXSTR); 
qsnprintf(arrptr[2] , MAXSTR, "%s", buffer); 

generate_disasm_line(cmd.ea, buffer, sizeof(buffer)); 
tag_remove(buffer, buffer, sizeof(buffer)) ; 
qsnprintf (arrptr[3], MAXSTR, "%s", buffer); 

return; 

} 

/************************************************************************** 

* Function: icEnter 

~k 

* This is a standard callback in the choose2() SDK call. This function 
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* is called when the user pressed Enter or Double-Clicks on a line in 

* the chooser list. 

**************************************************************************^ 
void idaapi icEnter(void * obj,ulong n) 

{ 

indirect_t curr_indirect; 
netnode *node = (netnode *)obj; 

node->supval(n, &curr_indirect, sizeof(curr_indirect)); 

jumpto(curr_indirect.caller); 

return; 

} 

/************************************************************************** 

* Function: icDestroy 

* 

* This is a standard callback in the choose2() SDK call. This function 

* is called when the chooser list is being destroyed. Resource cleanup 

* is common in this function. In this case any resource cleanup is 

* handled by register_event() . 

**************************************************************************j 

void idaapi icDestroy(void* obj) 

{ 

netnode *node = (netnode *)obj; 
msg("\"%s\" window closed\n", icTitle); 
register_event(E_DWCALL); 
return; 

} 

j************************************************************************** 

* Function: size 

* 

* This is a standard callback in the choose2() SDK call. This function 

* returns the number of lines to be used in the chooser list. 

**************************************************************************j 

ulong idaapi size(void* obj) 

{ 

netnode *node = (netnode *)obj; 
return getobjcount(node) ; 

} 
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* Function: createlndirectCallWindow 

~k 

* A wrapper around choose2() API. 'Generic list chooser (n-column )' 

* This sets up the callbacks and necessary options. 

* NOTE: 1. Cannot free the "object to show" until chooser closes 

* 2. Cannot unload plugin until chooser closes, 

* removing callbacks. 

**************************************************************************/ 
void createlndirectCallWindow(netnode* calls) 


choose2(false, 

- 1 , - 1 , - 1 , - 1 , 
calls, 

qnumber(icHeader) , 

icWidths, 

size, 

icDescription, 

icTitle, 

- 1 , 

0 , 

NULL, 

NULL, 

NULL, 

NULL, 
icEnter, 
icDestroy, 

NULL, 

NULL); 


// non-modal window 

// position is determined by Windows 
// object to show 
// number of columns 
// widths of columns 

// function that returns number of lines 
// function that generates a line 
// window title 

// use the default icon for the window 
// position the cursor on the first line 
// "kill" callback 
// "new" callback 
// "update" callback 
// "edit" callback 

// function to call when the user pressed Enter 
// function to call when the window is closed 
// use default popup menu items 
// use the same icon for all line 


} 

^'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k'k 


* Function: ccDescription 

★ 


* This is a standard callback in the choose2() SDK call. This function 

* fills in all column content for a specific line. Headers names are 

* set during the first call to this function, when n == 0. 

* arg: arrptr is a char* array to the column content for a line. 

* arrptr[number of columns] 
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* arg: completedbp_t* is atruct: netnode* - points to all calls 

* bphitlist_t - indexes of hit calls 

* 

* ccDescription creates 7 columns based on the icHeader array 

**************************************************************************I 

void idaapi ccDescription(void *obj,ulong n,char * const *arrptr) 

{ 

completedbp_t* cbp = (completedbp_t*)obj; 
indirect_t curr_indirect; 

if ( n == 0 ) // sets up headers 

{ 

for ( int i=0; i < qnumber(ccHeader) ; i++ ) 
qstrncpy(arrptr[i], ccHeader[i], MAXSTR); 
return; 

} 

bphitlist_t& tmp = * (bphitlist_t*)cbp->callindex; 
ulong index = tmp[n-l]; 

if (!tmp.size()) // only needed if choose2 kill callback used 

return; // since it removes members 

char buffer[MAXSTR] ; 

cbp->calls->supval(index, &curr_indirect, sizeof(curr_indirect)); 
func_t* currFunc = get_func(curr_indirect.caller) ; 
ua_anaO(curr_indirect.caller) ; // 

// seg.addr 

get_nice_colored_name(curr_indirect.caller, arrptr[0], 

MAXSTR, CNAMEOPT); 

get_short_name(BADADDR, currFunc->startEA, buffer, MAXSTR); 
qsnprintf (arrptr[1], MAXSTR, "%s", buffer); 

if (curr_indirect .flags & XRSETFLAG) 

qstrncpy(arrptr[2], "x", MAXSTR); // made a cross reference 
else 

qstrncpy(arrptr[2], MAXSTR); 

// get instruction disasm, remove color info 
generate_disasm_line(cmd.ea, buffer, sizeof(buffer) ); 
tag_remove(buffer, buffer, sizeof(buffer)) ; 
qsnprintf (arrptr[3], MAXSTR, "%s", buffer); 
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if (curr_indirect .flags & XSEGFLAG) 

qstrncpy(arrptr[4], "x", MAXSTR); // cross segment reference 

else 

qstrncpy(arrptr[4], MAXSTR); 

get_nice_colored_name(curr_indirect.target, 

arrptr[5], MAXSTR, CNAMEOPT); 

currFunc = get_func(curr_indirect.target); 

//demangles fname 

get_short_name(BADADDR, currFunc->startEA, buffer, MAXSTR); 
qsnprintf(arrptr[6] , MAXSTR, "%s", buffer); 

return; 

} 

/kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 

* Function: ccEnter 

k 

* This is a standard callback in the choose2() SDK call. This function 

* is called when the user pressed Enter or Double-Clicks on a line in 

* the chooser list. 

k-kk-kkk-k-k-k-k-k-kk-k-k-k-k-k-k-kkk-kk-k-k-k-kk-k-kk-k-kk-k-kk-kkk-k-k-k-k-k-kkk-k-k-k-kk-kkk-k-k-k-kk-kk-k-kk-k-kk-kkk-k I 

void idaapi ccEnter(void * obj,ulong n) 

{ 

completedbp_t* cbp = (completedbp_t*)obj; 
bphitlist_t &tmp = * (bphitlist_t*)cbp->callindex; 
indirect_t curr_indirect; 
ulong index = tmp[n-l]; 

cbp->calls->supval(index, &curr_indirect, sizeof(curr_indirect)); 

jumpto(curr_indirect.caller); 

return; 

} 

j •k-kkk-kk-kkk-kkk-kk-kkk-kkk-kkkkk-kkkkk-kkkkkk-kkkkk-kkk-kkk-kk-kkk-kkk-kkkkk-kkk-kk-kkk-kk-k-kkk 

* Function: ccDestroy 

k 

* This is a standard callback in the choose2() SDK call. This function 

* is called when the chooser list is being destroyed. Resource cleanup 

* is common in this function. In this case any resource cleanup is 

* handled by register_event() . 

kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkj 
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void idaapi ccDestroy(void* obj) 

{ 

completedbp_t* cbp = (completedbp_t*)obj; 
msg("\"%s\" window closed\n", ccTitle); 
register_event(E_DWXREFS) ; 
return; 

} 

/************************************************************************** 

* Function: ccSize 

* 

* This is a standard callback in the choose2() SDK call. This function 

* returns the number of lines to be used in the chooser list. 

************************************************************************** j 

ulong idaapi ccSize(void* obj) 


completedbp_t* cbp = (completedbp_t*)obj; 
return cbp->callindex->size() ; 

} 

/************************************************************************** 

* Function: createCompletedBpWindow 

* 

* A wrapper around choose2() API. 'Generic list chooser (n-column )' 

* This sets up the callbacks and necessary options. 

* NOTE: 1. Cannot free the "object to show" until chooser closes 

* 2. Cannot unload plugin until chooser closes, 

* removing callbacks. 

************************************************************************** j 

void createCompletedBpWindow(netnode* calls, bphitlist_t* bplist) 


completedbp_t* bp = new completedbp_t; 
bp->calls = calls; 
bp->callindex = bplist; 


choose2(false, 

- 1 , - 1 , - 1 , - 1 , 
bp, 

qnumber(ccHeader) , 

ccWidths, 

ccSize, 

ccDescription, 


// non-modal window 

// position is determined by Windows 
// object to show 
// number of columns 
// widths of columns 

// function that returns number of lines 
// function that generates a line 
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ccTitle, 

- 1 , 

0 , 

NULL, 

NULL, 

NULL, 

NULL, 
ccEnter, 
ccDestroy, 
NULL, 

NULL); 


// window title 

// use the default icon for the window 
// position the cursor on the first line 
// "kill" callback 
// "new" callback 
// "update" callback 
// "edit" callback 

// function to call when the user pressed Enter 
// function to call when the window is closed 
// use default popup menu items 
// use the same icon for all line 


} 


* Function: requestSetBps 

* 


* requests all our breakpoints be set, then run_requests 

************************************************************************** I 

void requestSetBps(netnode* node) 


indirect_t my_indirect; 

long no_calls = getnodesize(node); 

msg("requestSetBps size: %x\n", no_calls); 

for (int i = 1; i < no_calls; ++i) 

{ 

node->supval(i, &my_indirect, sizeof(my_indirect)); 
request_add_bpt(my_indirect.caller) ; 

} 

run_requests(); 
return; 

} 

I************************************************************************** 

* Function: requestDelBps 

ft 

* requests all our breakpoints be deleted, caller calls run_requests 
**************************************************************************^ 
void requestDelBps(netnode* node) 


indirect_t my_indirect; 

long no_calls = getnodesize(node) ; 
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msg("requestDelBps size: %x\n", no_calls); 
for (int i = 1; i < no_calls; ++i) 

{ 

node->supval(i, &my_indirect, sizeof(my_indirect) ) ; 
request_del_bpt(my_indirect.caller) ; 

} 

return; 

} 

* Function: setBps 

•k 

* set all our breakpoints 

**************************************************************************/ 

void setBps(netnode* node) 

{ 

indirect_t my_indirect; 

long no_calls = getnodesize(node); 

msg("setBps size: %x\n", no_calls) ; 
for (int i = 1; i < no_calls; ++i) 

{ 

node->supval(i, &my_indirect, sizeof(my_indirect)); 
add_bpt(my_indirect.caller); 

} 

return; 

} 

/kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k 

* Function: delBps 

* delete all our breakpoints 

kkkkkkkkkkkkkkkkkkk-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-kJ 

void delBps(netnode* node) 

{ 

indirect_t my_indirect; 
long no_calls = getnodesize(node) ; 
msg("delBps size: %x\n", no_calls); 
for (int i = 1; i < no_calls; ++i) 

{ 

node->supval(i, &my_indirect, sizeof(my_indirect)); 
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del_bpt(my_indirect.caller); 

} 

return; 

} 

/************************************************************************** 

* Function: setTargetXref 

★ 

* This function serves two purposes. First decides whether to add the 

* call to the completed call/bp list. It also can create the cross 

* reference between the caller and the target. 

**************************************************************************j 

void setTargetXref(dbgOptions* myDbg,long index,indirect_t* mylndirect) 

{ 

bphitlist_t* entry = myDbg->bplist; 
ulong options = myDbg->options; 
ea_t from = mylndirect->caller; 
ea_t to = mylndirect->target; 
short Sflags = mylndirect->flags; 
segment_t* from_seg = getseg(from); 
segment_t* to_seg = getseg(to); 

if (from_seg == to_seg) 

{ 

if (options & MAKE_XREFS) 

{ 

flags |= XRSETFLAG; 
if (flags & JMPSETFLAG) 

add_cref (from, to, (cref_t) (fl_JN | XREF_USER) ) ; 
else 

add_cref (from, to, (cref_t) (fl_CN | XREF_USER) ) ; 

} 

entry->push_back(index); 

} 

else // cross segment 

{ 

if (to_seg != NULL && ! (to_seg->is_ephemeral_segm())) 

{ 

flags |= XSEGFLAG; 
if (options & MAKE_XS_XREFS) 
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{ 


flags |= XRSETFLAG; 
if (flags & JMPSETFLAG) 


add 

else 

cref(from. 

to. 

(cref 

_t) 

(fl_JF | 

XREF USER) 

add 

cref(from. 

to. 

(cref 

_t) 

(fl_CF 

XREF USER) 


} 

if(options & DISPLAY_XS_BPS) 

{ 

entry->push_back(index); 

} 

} 

} 

} 

/************************************************************************** 

* Function: addVTable 

* 

* Determines if vtable is considered valid. A new vtable is added to 

* the vtable netnode. If the vtable already exists. The offset is 

* checked against the largest offset recorded for the vtable. 

************************************************************************** I 

void addVTable(dbgOptions* myDbg, ea_t vtaddr, indirect_t* mylndirect) 

{ 

ea_t from = mylndirect->caller; 
ea_t to = mylndirect->target; 
ea_t offset = myIndirect->offset; 

segment_t* from_seg = getseg(from); 
segment_t* vt_seg = getseg(vtaddr); 
netnode* vtables = myDbg->vtables; 
ulong options = myDbg->options; 

if (offset || (options & INC_NONOFF_CALLS)) 

{ 

if (from_seg != vt_seg) // only documenting vtables in from_seg 

{ 

return; 

} 

if ((get_first_dref_to(vtaddr) == BADADDR) | 

(get_first_dref_from(vtaddr) == BADADDR)) 
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{ 

msg("%x to %x , probably jump table, not vtable [%x]\n", 
from, to, vtaddr); 

} 

else // considered a valid vtable 


ulong tmp = vtables->altval(vtaddr) ; 
if (tmp == 0) // new vtable 

{ 

vtable_t my_vtable; 

int vtable_counter = getnodesize(vtables); 
my_vtable.baseaddr = vtaddr; 

my_vtable.largestOffset = myIndirect->offset; 
vtables->altset(vtaddr, vtable_counter); 
vtables->supset(vtable_counter++, &my_vtable, 
sizeof(my_vtable)); 
setnodesize(vtables, vtable_counter); 

msg("%x NEW VTABLE caller: %x , to: %x\n", vtaddr, from, to); 

} 

else // vtable already defined 


vtable_t tmpVtable; 

vtables->supval(tmp, stmpVtable, sizeof(tmpVtable)); 

// new offset > old offset 

if (myIndirect->offset > tmpVtable.largestOffset) 

{ 

tmpVtable.largestOffset = myIndirect->offset; 
vtables->supset(tmp, stmpVtable, sizeof(tmpVtable)); 

} 

} 

} 

} 

} 

* Function: callback 

* The debugger calls this function when handling any HT_DBG events. 

* The dbgOptions structure is passed to this function allowing the use 

* of previously defined data structures and user options. 
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* callback handles 3 types of HT_DBG events 

* 


* dbg_bpt - All breakpoints are handled here. The bp address 

* is checked to be ours. If not the the process is 

* suspended. Otherwise: 

* The instruction is call [eax] with or without an 

* offset OR anything else. 

* 

* For everything else 'step into' is requested. 

* The current bp addresses is saved in last_bp 

* for the step_into handler 

* 


With the instruction decoded, both the base and 
target can be calculated. 
addVTable() & setTargetXref() process if 
vtables and cross references are made. The 
indirect_t obj is saved with updates. 
continue_process() is called 


dbg_step_into - All step_into events are handled here. last_bp 
is checked. For user caused step_into event 
suspend_process() is called. 

setTargetXref() deltemines if cross references 
are made. The indirect_t obj is saved with updates. 
continue_process() is called 


* dbg_process_exit - This event signifies that the debugger is 

* shutting down. Brealpoints are cleared and depending on 

* options, up to three chooser list windows are opened. 

************************************************************************** I 

int idaapi callback(void* user_data,int notification_code,va_list va) 


dbgOptions* my_dbg = (dbgOptions*)user_data; 

netnode* calls = my_dbg->calls; 

ulong options = my_dbg->options; 

static ea_t last_bp = BADADDR; 

ea_t from = BADADDR; 

ea_t vtaddr = BADADDR; 

ea_t to = BADADDR; 

regval_t regval; 
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switch (notification_code) 

{ 

case dbg_bpt: 

{ 

va_arg(va, tid_t); 

from = va_arg(va, ea_t); 

long index = calls->altval(from); 

if (index == 0) 

{ 

// not one of our breakpoints 

msg("%x not mine options:0x%x", from, options); 
suspend_process(); 
return 0; 

} 

indirect_t my_indirect; 

calls->supval(index, &my_indirect, sizeof(my_indirect) ) ; 

// check for call [reg] or call [reg + offset] 
if (my_indirect.call_reg == none) 

{ 

last_bp = from; 
request_del_bpt(from) ; 
request_step_into(); 
run_requests(); 
break; 

} 

get_reg_val(regname[my_indirect.call_reg] , &regval); 
vtaddr = (ea_t)regval.ival; 

// flushes possibly stale memory cache 
invalidate_dbgmem_contents((ea_t)regval.ival, 

0x100 + my_indirect.offset) ; 
to = get_long(my_indirect.offset + vtaddr); 
my_indirect.target = to; 

addVTable(my_dbg,vtaddr, &my_indirect); 
setTargetXref(my_dbg, index, &my_indirect); 

// save completed indirect 

calls->supset(index, &my_indirect, sizeof(my_indirect)); 
del_bpt(from); 
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continue_process(); 
break; 

} 

case dbg_step_into: 

{ 

from = last_bp; 
if (from == BADADDR) 

{ 

msg("not mine\n"); 
suspend_process(); 
return 0; 

} 

long index = calls->altval(from); 
get_reg_val("EIP", Sregval); 
to = (ea_t)regval.ival; 
indirect_t my_indirect; 

calls->supval(index, &my_indirect, sizeof(my_indirect)); 
my_indirect.target = to; 

setTargetXref(my_dbg, index, &my_indirect); 

// save completed indirect 

calls->supset(index, &my_indirect, sizeof(my_indirect)); 
last_bp = BADADDR; 
continue_process() ; 
break; 

} 

case dbg_process_exit: 

{ 

unhook_from_notification_point(HT_DBG, callback, user_data) 
requestDelBps(calls); 
run_requests(); 
register_event(E_PROCEXIT); 

if (options & DISPLAY_INCALLS) 

{ 

createlndirectCallWindow(calls); 

} 

if (options & DISPLAY_BPS) 

{ 

createCompletedBpWindow(calls, my_dbg->bplist); 

} 
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if (options & DISPLAY_VTABLES) 

{ 

createVTableWindow(my_dbg->vtables) ; 

} 

break; 

} 

default: 
break; 

} 

return 0; 

} 

/************************************************************************** 

* Function: getnodesize 

★ 

* returns size (including location 0) 

**************************************************************************j 

long getnodesize(netnode* node) 

{ 

return node->altval(NODE_COUNT); 

} 

j************************************************************************** 

* Function: getobjcount 

* 

* returns number of items in the netnode not counting invalid slot 0 

* see data structure documentation at top of file 

**************************************************************************j 

long getobjcount(netnode* node) 

{ 

return node->altval(NODE_COUNT)-1; 

} 

/************************************************************************** 

* Function: setnodesize 

* 

* store netnode size 

**************************************************************************j 

bool setnodesize(netnode* node, long size) 


return node->altset(NODE_COUNT, size); 

} 
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* Function: filllndirectObj 

* 

* Determines if instruction is call [reg+offset], call [reg], or other 

* Fills in the indirect_t struct. 

**************************************************************************j 

void filllndirectObj(indirect_t Scurrcall) 

{ 

currcall.caller = cmd.ea; 
currcall.target = BADADDR; 
currcall.call_reg = none; 
currcall.offset = 0; 
if (cmd.itype == NN_callni) 

{ 

// need a single opcode 
ushort no_operands = 0; 
while(no_operands < UA_MAXOP && 

cmd.Operands[no_operands].type != o_void) 

{ 

no_operands++; 

} 

if (no_operands == 1) 

{ 

if (cmd.Operands[0].type == o_phrase) 

{ 

currcall.call_reg = cmd.Operands[0].reg; 

} 

else if (cmd.Operands[0].type == o_displ) 

{ 

currcall.call_reg = cmd.Operands[0].reg; 
currcall.offset = cmd.Operands[0].addr; 

} 

} 

} 

else if (cmd.itype & NNJMPxI) // jmp? 

{ 

currcall .flags |= JMPSETFLAG; 

} 
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/ '********************************************** 

* Function: findlndirectCalls 

* 

* This function through a segment for indirect 

* NN_callfi, NN_callni, NN_jmpfi, NN_jmpni 

* then it pkgs it in a inidirect_t struct and 
*********************************************** 

void findlndirectCalls(segment_t* seg, netnode* 

{ 

ea_t addr = seg->startEA; 
ulong counter = getnodesize(node); 
while ((addr < seg->endEA) && (addr != BADADDR)) 

{ 

flags_t flags = getFlags (addr) ; 
if (isHead (flags) && isCode (flags) ) 

{ 

if (ua_anaO(addr) != 0) 

{ 

switch (cmd.itype) 


**************************** 

calls and jmps 

stores in the netnode 

*************************** j 

node) 


case NN_callfi: 
case NN_callni: 
case NN_jmpfi: 
case NN_jmpni: 


if (get_first_f cref_f rom (cmd. ea) 
get_first_dref_f rom (cmd. ea) == 


— BADADDR & & 

BADADDR) //no fwd xref 


indirect_t currcall; 
filllndirectObj(currcall) ; 

node->altset(cmd.ea, counter); // altval keyed by addr 
node->supset(counter++, Scurrcall, sizeof(currcall)); 

} 

break; 

} 

default: 
break; 

} 

} 
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} 

addr = next_head(addr, seg->endEA); 

} 

setnodesize(node, counter); 
return; 

} 

void closeListWindows(void) 

{ 

close_chooser(icTitle) ; 
close_chooser(ccTitle) ; 
close_chooser(vtTitle) ; 

} 

/************************************************************************** 

* Function: register_event 

~k 

* This function serves as an interface to three semaphores in the form 

* of event messages. IDA Pro is single threaded and is non reentrant. 

* True concurrency requirements such as mutexes and atomic operations 

* are not needed. 

* The caller reports an event and this function adjusts the semaphores 

* and can release resources when needed. 

* semaphores are tied to the 

* netnode* calls - all indirect calls 

* netnode* vtables - all vtables 

* bphitlist_t* bplist - bp hits, an index list into 

* netnode* call 

* ★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★it j 

void register_event(ulong rEvent) 

{ 

static long dbgState = 0; 
static long semcall = 0; 
static long semxref = 0; 
static long semvtable = 0; 
switch (rEvent) 

{ 

case E_START: 

{ 

closeListWindows(); 
if (gDbgOptions.calls) 
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{ 

gDbgOptions.calls->kill(); 

} 

if (gDbgOptions.vtables) 

{ 

gDbgOptions.vtables->kill() ; 

} 

if (gDbgOptions.bplist) 

{ 

gDbgOptions.bplist->~qvector(); 

} 

semcall = semxref = dbgState = semvtable = 0; 
break; 

} 

case E_CANCEL: 

{ 

semcall = semxref = dbgState = semvtable = 0; 

gDbgOptions.calls->kill() ; 

gDbgOptions.vtables->kill() ; 

gDbgOptions.bplist->~qvector() ; 

break; 

} 

case E_OPTIONS: 

{ 

if ( (~gDbgOptions.options) » 15) 

{ 

dbgState++; 
semcall++; 
semxref++; 
semvtable++; 

} 

if (gDbgOptions.options & DISPLAY_INCALLS) 

{ 

semcall++; 

} 

if (((gDbgOptions.options & DISPLAY_BPS) » 1) && dbgState) 

{ 

semcall++; 
semxref++; 

} 
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} 

case 

{ 

} 

case 

{ 


} 

case 

{ 


} 

case 

{ 


if (((gDbgOptions.options & DISPLAY_VTABLES) » 

{ 

semvtable++; 

} 

break; 

E_HOOKFAIL: 

dbgState = semvtable = semxref = 0; 
break; 

E_PROCFAIL: 

// note: call window may be open 
delBps(gDbgOptions.calls) ; 
semcall—; 

dbgState = semvtable = semxref = 0; 

unhook_from_notification_point(HT_DBG, callback, 

break; 

E_DWCALL: 

semcall—; 
if (!semcall) 

{ 

gDbgOptions.calls->kill() ; 

} 

break; 

E_DWXREFS: 

semxref—; 
semcall—; 
if (!semcall) 

{ 

gDbgOptions.calls->kill() ; 

} 

if(!semxref) 

{ 

gDbgOptions.bplist->~qvector() ; 

} 


5) && dbgState) 


SgDbgOptions) ; 
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break; 

} 

case E_DWVTABLE: 

{ 

semvtable—; 
if (!semvtable) 

{ 

gDbgOptions.vtables->kill() ; 

} 

break; 

} 

case E_PROCEXIT: 

{ 

dbgState = 0; 
semcall—; 
semvtable—; 
semxref—; 
if(!semxref) 

{ 

gDbgOptions.bplist->~qvector() ; 

} 

if (!semcall) 

{ 

gDbgOptions.calls->kill() ; 

} 

if (!semvtable) 

{ 

gDbgOptions.vtables->kill() ; 

} 

break; 

} 

default: 


msg("ERROR UNKNOWN EVENT\n") ; 

msg("%s dbg:%d scall:%d sxref:%d svtable:%d \n", 

"ERROR", dbgState, semcall, semxref, semvtable); 
break; 

} 

} 

} 
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/ 'k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k 

* Function: run 

* 

* run is a plugin_t function. It is executed when the plugin is run. 

* This function brings up the UI, collects data and sets the debugger 

* callback. 

* arg - defaults to 0. It can be set by a plugins.cfg entry. In this 

* case the arg is used for debugging/development purposes 


* ;plugin displayed name 

filename 

hotkey 

arg 

* indirectCalls dbg 

indirectCalls 

Alt-F8 

0 

* indirectCalls unload 

indirectCalls 

Alt-F9 

415 


* 


* Thus Alt-F9 runs the plugin with an option that will unload it. 

* This allows (edit/recompile/copy) cycles. 
*************★**************************************************★*********/ 
void run(int arg) 

{ 

char nodename_calls[] = "$ indirect calls"; 
char nodename_vtables[] = "$ vtables"; 
ea_t curraddr = get_screen_ea(); 
segment_t* my_seg = getseg(curraddr) ; 
char* format; 

short checkbox = DISPLAY_INCALLS | DISPLAY_BPS I DISPLAY_VTABLES; 

short radiobutton = 0; 

int start_status; 

register_event(E_START) ; 

if(arg == 415) 

{ 

PLUGIN.flags |= PLUGIN_UNL; 
msg ("Unloading plugin...\n") ; 
return; 

} 

netnode* calls = new netnode; 
netnode* vtables = new netnode; 
bphitlist_t *hitlist = new bphitlist_t; 
if (calls->create(nodename_calls) == 0) 

{ 

calls->kill(); 

msg("ERROR: creating netnode %s\n", nodename_calls); 
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return; 

} 

if (vtables->create(nodename_vtables) == 0) 

{ 

msg("ERROR: creating netnode %s\n", nodename_vtables) ; 
calls->kill() ; 
vtables->kill() ; 
return; 

} 

calls->altset(NODE_COUNT, 1) ; // position 0 is not used 

vtables->altset(NODE_COUNT,1); // position 0 is not used 

findlndirectCalls (my_seg, calls); // finds jmps/calls 

ulong format_size = sizeof(preformat)+9; 
format = (char*)qalloc(format_size); 

qsnprintf(format, format_size, preformat, getobjcount(calls)); 

int ok = AskUsingForm_c(format, Sradiobutton, Scheckbox); // UI 

gDbgOptions.calls = calls; 

gDbgOptions.vtables = vtables; 

gDbgOptions.bplist = hitlist; 

gDbgOptions.options = checkbox; 

register_event(E_OPTIONS) ; 
if ( ! ok) 

{ 

msgC'user canceled, exiting, unloading\n"); 
register_event(E_CANCEL); 

PLUGIN.flags |= PLUGIN_UNL; 
return; 

} 

// debugger closing this window, now only open for non debugger 
if ( (checkbox & DISPLAY_INCALLS) && (radiobutton == 1)) 

{ 

createlndirectCallWindow(calls) ; 

} 

if (radiobutton == 1) 

return; // only collect data 

// the hook is created here, callback() will receive HT_DBG 
// events only. gDbgOptions is passed to callback() 

// it is global so termination funcs have access 

if ( !hook_to_notification_point(HT_DBG, callback, &gDbgOptions)) 
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{ 

warning("Could not hook to notification point\n"); 
register_event(E_HOOKFAIL); 
return; 

} 

requestSetBps(calls); 

start_status = start_process(NULL, NULL, NULL); 
if (start_status == 1) // SUCCESS 

{ 

msg("process started...\n") ; 
return; 

} 

else if (start_status == -1) 

{ 

warning("Sorry, could not start the process"); 

} 

else 

{ 

msg("Process start canceled by user\n"); 

} 

register_event(E_PROCFAIL); 
return; 

} 

/************************************************************************** 

* Function: init 

* 

* init is a plugin_t function. It is executed when the plugin is 

* initially loaded by IDA 

************************************************************************** j 

int init(void) 

{ 

if (ph.id != PLFM_386) // intel x86 

return PLUGIN_SKIP; 
return PLUGINJDK; 

} 

/************************************************************************** 

* Function: term 

* 

* term is a plugin_t function. It is executed when the plugin is 
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* unloading. Typically cleanup code is executed here. 

* The unhook is called as a safety precaution. 

* The windows are closed to remove the choose2() callbacks 

void term(void) 


unhook_from_notification_point(HT_DBG, callback, SgDbgOptions); 

closeListWindows(); 

return; 

} 

char comment!] = "indirectCalls"; 

char help[] = "This plugin looks\nfor indirect\ncalls\n"; 
char wanted_name[] = "indirectCalls"; 
char wanted_hotkey[] = "Alt-F7"; 

/* defines the plugins interface to IDA */ 
plugin_t PLUGIN = 


IDP INTERFACE VERSION, 


0 , 

init, 

term, 

run, 

comment, 

help, 

wanted_name, 

wanted_hotkey 


// plugin flags 
// initialize 

// terminate, this pointer may be NULL. 

// invoke plugin 
// comment about the plugin 
// multiline help about the plugin 
// the preferred short name of the plugin 
// the preferred hotkey to run the plugin 


Plug-in Development 
and Debugging Strategies 

This section provides some useful strategies to help writing and debugging plug-ins. The 
Visual Studio debugger works relatively well and is convenient. The debugger can attach 
and detach to the IDA Process. 

Create a new IDA Development Directory 

Copy the IDA Pro install directory to a new location, leaving the original directory intact. 
Choose something short which does not require a lot of typing. For example use: 
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C:\ida_dev 

Go into the plugin directory, in this case C:\ida_dev\plugins, and create a new 
directory called plugin_backup. Copy the contents of the plugin directory into the 
plugin_backup directory. Next begin deleting any plug-ins that are not required for the 
development of the current plug-in. For example if developing a 32 bit plug-in, all the 
64 bit plug-ins can be removed. Make sure the keep the debuggers if you are developing 
a plug-in that uses the debugger. 

The removal of the plug-ins serves multiple purposes. 

■ The message window will contain less extraneous debug messages when using the 
—z debug option, which will be discussed shortly. 

■ Removing the plug-ins also frees potential hotkeys. We may want to set multiple 
hotkeys to pass different arguments to the plug-in being developed. 

■ Startup time decreases without the initialization of unnecessary plug-ins. 

Editing Configuration Files 

Edit configuration files with testing in mind rather than normal operation. This means 
removing unnecessary hot key bindings and adding others that may be useful. The following 
are located in idagui.cfg: 

Using an Unpacked Database 

IDA can operate on the unpacked database which speeds up starting and stopping of the process. 
When developing a plug-in never use an IDB file without making a backup. In order to operate 
with unpacked databases do the following: 

1. Copy the IDB to a new directory. 

2. Make a batch file in the same directory. This file will execute the development 
copy of IDA. This also allows setting command line arguments. An example idadev. 
bat file could contain the following: 

C:\ida_dev\idag.exe myidb.idb 

3. Run the batch file. Exit and select Don’t pack database. You will get a warning, 
but select Yes. The IDB file is no longer in the directory. At this point options can 
be changed to remove the warnings. 

4. Set the following options in idagui.cfg: 


ASK EXIT UNPACKED 

= NO 

// 

Ask confirmation 

if 

the user 



// 

wants to exit the 

database without 



// 

packing it 



ASK EXIT 

= NO 

// 

Ask confirmation 

if 

the user 



// 

wants to exit 




www.syngress.com 


5. Optionally you can assign a hotkey to “Abort”. 

6. Set the following options in ida.cfg: 
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PACK DATABASE 

= 0 

// 

0 

- don't pack at 

all 



// 

1 

- pack database 

(store) 



// 

2 

- pack database 

(deflate) 


The advantages of working with an unpacked database are faster startup and shutdown 
times. 

Note that these options should only be set for the development copy of IDA. The 
options are not generally recommended. 

Enabling Exit without Saving 

An alternative strategy is to never save the IDB while developing. The unpacked method 
will still save the files. If your plug-in crashes the state of the files and database may be 
unknown. To facilitate operating without saving do the following: 

1. Copy the IDB to a new directory. 

2. Make a batch file in the same directory. This file will execute the development 
copy of IDA. This also allows setting command line arguments. An example idadev. 
bat file could contain the following: 

C:\ida_dev\idag.exe myidb.idb 

3. Assign a hotkey to “Abort” 

"Abort" = "Alt-Z" // Abort IDA, don't save changes 

When you execute “Abort” a confirmation dialog will come up. There appears to be no 
options to prevent it, but pressing Y will exit. 

This strategy has its advantages; the primary one is having a known starting IDB every 
time in the testing cycle. The downside is somewhat slower start up times. The shutdown 
time is negligible since IDA doesn’t save and pack the database. The amount of delay 
depends on the size of the IDB. 

Plug-in Arguments 

Plug-ins can be passed arguments. This can be used to control and change the plug-ins 
behavior. The plugis.cfg file defines hotkeys and arguments to plug-ins. 

The IDA API does not allow the unloading of a plug-in. Most non trivial plug-ins will 
establish callbacks or hooks and remain in memory. This prevents an updated recompiled 
copy of the plug-in from overwriting the current one. One could exit IDA, but there is a 
workaround using plug-in arguments. The following is from a plugins.cfg file: 

indirectCalls indirectCalls Alt-F8 0 

indirectCalls unload indirectCalls Alt-F9 415 
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The corresponding code to handle the argument is: 

if(arg == 415) 


PLUGIN.flags |= PLUGIN_UNL; 
msg(" Unloading plugin...\n") ; 
return; 

} 

The PLUGIN_UNL flag can be set anytime but IDA checks it upon exit of the run 
function.The plug-in is called with the argument 415, the PLUGIN_UNL flag is set. The 
plug-in should ensure that it unhooks from any notification as well as removing any call¬ 
backs. The plug-in using the preceding code performs unhooks in the term function. 

Arguments can be used for other things besides unloading the plug-in. An argument 
could be defined to set a global debug flag. Multiple output functions could exist. For exam¬ 
ple a certain arg could dump results to the message window, while a different arg can create a 
chooser list box. The argument can be sent from IDC as well, using the Run Plugin function. 
RunPlugin("indirectCalls", 415); 

Scripting to Help Plug-in Development 

Scripting is very useful to test concepts or prototype before writing a plug-in. In particular 
IDAPython can be very useful since it wraps many of the API calls. IDC can be used as well. 
Although it lacks some of the more advanced APIs, IDC is always available. 

During the development and testing of the indirect calls plug-in, IDC scripts were 
used. The plug-in uses cross reference data for much of its logic. In order to test and verify 
that both the plug-in and theories were sound, a script was written. The following is the 
script. 

#include <idc.idc> 
static decode_xtype(xtype) 

{ 

if (xtype & XREF_USER) 


Message("XREF_USER") ; 
xtype = xtype & ~XREF_USER; 

} 

if (xtype == fl_CF) 

Message ("fl_CF Call Far") ; 
else if (xtype == fl_CN) 

Message ("fl_CN Call Near") ; 
else if (xtype == fl_JF) 

Message ("fl_JF Jump Far"); 
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else if (xtype == fl_JN) 

Message ("fl_JN Jump Near") ; 

else if (xtype == fl_F) 

Message ("fl_F Ordinary flow") ; 
else if (xtype == dr_0) 

Message("dr_0 Offset"); 
else if (xtype == dr_W) 

Message("dr_W Write"); 
else if (xtype == dr_R) 

Message("dr_R Read" ); 
else if (xtype == dr_T) 

Message("dr_T Text (names in manual operands)"); 
else if (xtype == dr_I) 

Message("dr_I Informational") ; 

} 

static lookup_from_ref(void) 

{ 

auto from, current_code, current_data, no_cxrefs, no_dxrefs; 
from = ScreenEA(); 
no_cxrefs = 0; 
no_dxrefs = 0; 

Message("%x [from] xrefs\n", from); 
current_code = RfirstO (from) ; 
while(current_code != BADADDR) 

{ 

no_cxrefs++; 

Message(" %x CODE (0x%x) ",current_code, XrefTypeO); 

decode_xtype(XrefType() ) ; 

Message("\n") ; 

current_code = RnextO(from, current_code); 

} 

current_data = Dfirst (f rom) ; 
while(current_data != BADADDR) 

{ 

no_cxrefst+; 

Message(" %x DATA (0x%x) ",current_data, XrefTypeO); 

decode_xtype(XrefType() ) ; 

Message("\n") ; 

current_data = Dnext(from, current_data); 

} 
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if ( (no_cxrefs + no_dxrefs) == 0) 

Message(" NONE\n"); 

} 

static lookup_to_ref(void) 

{ 

auto to, current_code, current_data, no_cxrefs, no_dxrefs 
to = ScreenEAO; 

no_cxrefs = 0; 
no_dxrefs = 0; 

Message("%x [to] xrefs\n", to); 
current_code = RfirstBO (to) ; 
while(current_code != BADADDR) 

{ 

no_cxrefs++; 

Message(" %x CODE (0x%x) ",current_code, XrefTypeO ); 

decode_xtype(XrefType() ) ; 

Message ("\n"); 

current_code = RnextB0(to, current_code); 
if (current_code != BADADDR && no_cxrefs > 7) 

{ 

Message (" TOO MANY (%d) CODE xrefs no_cxrefs); 

current_code = BADADDR; 

} 

} 

current_data = DfirstB (to) ; 
while(current_data != BADADDR) 

{ 

no_dxrefs++; 

Message(" %x DATA (0x%x) ",current_data, XrefTypeO ); 

decode_xtype(XrefType()); 

Message ("\n"); 

current_data = DnextB(to, current_data); 
if (current_data != BADADDR && no_dxrefs > 7) 

{ 

Message (" TOO MANY (%d) DARA xrefs no_dxrefs) ; 

current_data = BADADDR; 

} 

} 

if ( (no cxrefs + no dxrefs) == 0) 
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Message(" NONE\n"); 

} 

static main(void) 

{ 

AddHotkey("Shift-F7", "lookup_to_ref "); 

AddHotkey("Shift-F8", "lookup_from_ref"); 

} 

The script binds hotkeys to the lookup functions. Code and data cross references are 
listed in the message window. While IDA includes xref.idc, the format was difficult to read 
quickly. The following is sample output from my_xref.idc, including a listing of the instruction 
it processed. 

.text:030BDF13 call dword ptr [eax+18h] ; my_xref.idc 

30bdfl3 [from] xrefs 

30cba2 5 CODE (0x13) fl_JN Jump Near 
30bdfl3 [to] xrefs 
NONE 

The script is loaded by ida.idc. When a script is included, main does not executed but the 
functions are available.Thus the hotkeys are bound within ida.idc, as shown in the following 
bit of code. 

#include <my_xrefs.idc> 
static main(void) { 

// 

// This function is executed when IDA is started. 

// 

// Add statements to fine-tune your IDA here. 

// 

AddHotkey("Shift-F7", "lookup_to_ref"); 

AddHotkey("Shift-F8", "lookup_from_ref"); 

} 

Loaders 

Toaders are responsible for recognizing file formats and creating appropriate segments. 
Analysis is generally performed by processor modules. Toaders as the name implies only 
load a binary into IDA. 

There are various processor modules with source code in the modules directory of the 
SDK. There are some publically released loaders. NSDLDR is a loader for Nintendo DS 
ROM files written by Dennis Elser (http://www.openrce.org/downloads/details/56/ 
NDSTDR).The loader is relatively simple and the code is easy to follow. 
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Loaders export the loaderjt structure which is defined in loader.hpp. 

struct loader t 


ulong 

version; 

// 

ulong 

flags; 

// 

accept file; 

// 



// 

load 

file; 

// 

save 

file; 

// 

move 

segm; 

// 

init 

loader options; 

// 


api version, should be IDP_INTERFACE_VERSION 
loader flags 

checks the input format. Shows up in the 

"load file" dialog box 

loads file into database 

can create output file from database 

moves segment for relocation or rebasing 

initialize user configurable options 


Processor Modules 

Processor modules perform the actual disassembly and analysis of the binary With over 
50 families of processors already supported, most of the major CPUs are covered. However 
many embedded devices do not have modules yet. Smaller scale devices are built for low cost 
and such have simpler architectures. These devices can range from standard microcontrollers 
to rare and limited run chips in audio and video equipment. If there is firmware, someone 
is going to reverse it. 

The SDK has source to many processor modules ranging from the ever popular Atmel AVR 
chip to the classic z80. Most of the modules use the same file naming convention for each of 
the main structures allowing for a compare and contrast between modules. The structures used 
by modules are defined in idp.hpp. The main structures are processor_t, asmjt, and instmctjt. 

Perhaps writing modules to decode tiny silicon is not to your liking. Modules can and 
have been written for virtual machines as well.VMs are becoming more popular everywhere 
from embedded devices to software protections and crackmes. Whether your interest is writ¬ 
ing a module for silicon or imaginary silicon, Rolf Rolles’ article Defeating HyperUnpackMe2 
With an IDA Processor Module is a must read. Appendix B in particular. (http://www.openrce. 
org/ articles/fuh_view/28) 


Third-party Scripting Plug-ins 

We aren’t limited to just writing IDC scripts or full plug-ins in C++. Third party scripting 
plug-ins provide an alternative. Often using SWIG they wrap many IDC and SDK functions. 

The use of a scripting language like Python and Ruby allow access to large libraries of 
code. Maybe more importantly they bring their nice built in data types. There are currently 
two choices for scripting languages. Python brought to us in the form of IDAPython and 
the second is Ruby as IdaRub. 
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The first scripting plug-in may have been IDAPerl, but it does not appear to be available 
for download or supported. 

IDAPython 

IDAPython (http://d-dome.net/idapython) is written by Gergely Erdelyi. It is a very popu¬ 
lar plug-in for IDA. New releases focus on coverage of wrapped functions and adding new 
SDK functions.The source code is available in a Darg repository. 

Supported Platforms 

IDAPython can run under the Windows or Linux. New test releases are reported to work 
under Mac OS X. 

Installation under Windows is fairly straightforward. IDAPython is available compiled 
against either Python 2.4 or 2.5. Unless you have specific reasons you want 2.4, install 
Python 2.5 (http://www.python.org/download/). 

Download the appropriate version of the plug-in. I generally use test releases as they will have 
more wrapped functions.Test releases are hosted here http://code.google.eom/p/idapython/. 

Unzip the package. Installation consists of copying files to the appropriate places. Copy the 
python directory to IDA Pro’s install directory. Copy python.plw from the plug-in directory 
to IDA Pro’s plug-in directory. The plug-in is now installed and will be ready to use the 
next time IDA is started. 

A function reference is available for download or online www.d-dome.net/idapython/ 
reference/. It is generated by epydoc directly from the source code. 

IDARub 

IDARub (http://www.metasploit.com/users/spoonm/idarub/) as implied by the name uses 
Ruby as its scripting language. IDARub is written by Spoonm. Ruby too has become popular 
for security tools, the most known being the Metasploit Framework (www.metasploit.com). 

The current version of IdaRub is 0.8 was released on August 1, 2006. Since it compiled 
against the 4.9 SDK, it will continue working with future versions of IDA. However new 
functions added to the SDK since 4.9 will not be available. 

While IDAPython is more popular and supported, there are features only available in 
IdaRub. Some of the feautures are: 

■ Remote network access 

■ Console 

■ Sweet demos 

Sebastian Porst wrote Rublib (http://www.the-interweb.com/serendipity/index.php?/ 
archives/91-RubLib-0.04.html) which is described as a high level API for IdaRub. The 
current version is 0.04 and it contains over 160 helper functions. 
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Frequently Asked Questions 

Q: Can I make a multithreaded plug-in? 

A: IDA Pro is very definitely single threaded. All access to the database would have to be 
serialized. There are some examples of multithreaded plug-ins. IdaRub written by 
Spoonm creates a hidden window and handler. Source code is available here: http:// 
www.metasploit.com/users/spoonm/idarub. 

Q: My plug-in outputs information to the message window. The message window seems to 
only hold 2000 lines, can I increase the size of the buffer? 

A: IDA can redirect the messages to a log file, if you set the IDALOG environmental 
variable., set IDALOG=mylog . txt 

Q: The list boxes are useful, but can I use the graphing engine for output? 

A: The SDK comes with a sample plug-in ugraph which creates a graph view. In the SDK 
graph.hpp contains the classes relating to graph creation. 
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stop execution, 89 
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code learning, 183 
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code traversal, 225 
collectData function, 251 
common object file format (COFF), 38 
file header for, 40 
compiler strings, 174, 175 
computer viruses, 195 
conditional jump registers 
call and ret instructions, 34 
jcc and jmp instructions, 31 
configuration files, process 
for editing, 301 

constructor method, for ECX register, 182 
control register 3 (CR3), 18 
control register 4 (CR4), 110 
C printf function, 202 
CStringArray, 180 
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data representation functions, 224 
DataRescue, 2 

dbg_process_exit, for termination of 
debugged process, 262 
debug exceptions 
EFLAG register, 113 
fault class, 112-113 
instruction breakpoint for, 113 
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trap class, data and I/O breakpoints, 113 
debuggers 

ability to perform, 88 
application setup, 96 
essential tool, 88 
exceptions, 91 
GNU project, 106 

hardware and software breakpoints, 89, 90 
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method for termination of, 176 

modules for, 229 

Oily dbg, Win32 user mode, 105 
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setup options, 93 

tracing, 91 

user mode us. kernel mode, 88 
Windbg, 104, 105 

debugger us. Interactive Disassembler Pro 
(IDA Pro), 2 
debugging 
form of, 88 
in IDA Pro, 92 

reverse engineering, use of, 94 
strategies for, 251 
tools for Windows, 104 
use of, 88 

debugging extensions (DE) flag, 
interpretations for. 111 
debug registers (DR), 89 
DR 0 thro DR 3, 110-112 
DR 6 thro DR 7, 111 
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ECX register, 179, 180 
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ELF (executable and linkable 
format), 38, 229 
ELF linux binaries, 73 


www.syngress.com 


Index 313 


end address parameter, 170 
entry point obscuring or EPO technique, 136 
ESI register, 178 
exceptions 
configuration, 102 
error analysis, 91 
executable and linkable format 
(ELF), 38 
attributes, 55 

common section table, 56-58 
dynamic linking, 64 
dynamic structure 
defined d_tag types, 61 
global offset table (GOT), 62—63 
header, 50-51 

procedure linkage table (PLT), 60, 63 
program header table, 58-60 
section content types, 53-55 
section header table array, 52-53 
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findlndirectCalls function, 257 
findMemcpy, 249 

flat memory model vs. segmented 
memory model, 15—16 
FLIRT signatures, loading files, 173 
function calling, 195, 196 
fu nction Search, 251 

G 

GDB (GNU project debugger), 106 
general purpose registers, 11-12, 176 
GetFuncAttr calls, 213 
GetFunctionAttr, 204 
GetProcAddress () 
hash value, 134 
pointer’s purpose, 135 
get_screen_ea, 238 

Glibc allocated memory blocks, 23 
Glibc free memory blocks, 24 


GNU project debugger (GDB), 106 
GS stack protection, 103 
GUI, 183 

H 

hard coded password, process for 
recovering, 85 
hardware breakpoints, 89 
heap corruption, checking for, 102, 103 
heap data structure, 22 
hexadecimal characters, 169 
hit marking 

breakpoint list or hit list, 155 
interactive disassembler (IDA) 
debuggers built-in mechanism, 156 
plug-in, 156 
python snippet, 158 
setting a trace point, 157 
short trace, 155 
picking message processing 
function, 153 
Pidgin example, 158-160 
protocol dumps for comparison, 161 
wpurple_read call, 161 
human anti-virus application, 177 

I 

IA-32 processor, 112, 113 
IDA 

API functionality, 250 
debugger vs. others, 130-131 
detection, 170 

development directory, process 
for creating, 301 
plug-ins, basics of, 227 
scripting, basics of, 200 
unpacker plug-in, 168 
versions of, 169 
IDAPython, 214, 308 
IDARub, 309 
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debugger functionality, 221-222 
language, 213 
libraries, 208 
scripting 

global variables for, 207 
language syntax for, 201 
method for function 
declarations in, 205 
sampling of IDC functions, 222 
immunity debugger, 105 
import address table (IAT), 136 
indirect call plug-in, 256, 271 
INI file, 197 

initialization callback, 184 
Interactive Disassembler Pro 
(IDA Pro), 2 
debugging, 92 
definition, 2 

dissemination of variables 
and arguments, 77 
instantmsgrs.exe, 3-4 
introduction of SDK version 4.9, 230 
layout of conditionals, 83 
methods for extending, 200 
plug-in wizard for, 232 
scripting and writing plug-ins, 200 
static analysis, 88 
WootBot variant, 3-4 
invalidate_dbgmem_contents function, 261 
I/O cycle in routine returns, 192 

L 

Last Stage of Delirium (LSD), 132 
library modules, list of, 173 
loaders, for recognizing file 
formats, 307 
loaderjt structure, 307 
load file, 172 
LoadLibraryA() call, 134 


M 

machine interface (MI), 106 
malicious software, 168 
malware skills, 167, 195 
memory corruption, detection, 102 
memory operands, 13 
memset() function, 71, 76 
message handling code, 196 
MFC 4.2, 175 
MFC DLL, 189 
MI (machine interface), 106 
module/plug-in resources, 227-230 
Molebox, 3 
mov instructions, 76 
MSF_HB:: Read St re a m function, 215 
MS Visual C++ runtime signature, 175 
multiple operating systems, degrees of 
difficulty, 166 

N 

Netcat client, 100 
NULL pointer, 181, 188 

O 

OEP (original entry point), 168 
opcodes, 9 

operating system, 103 
loaders, 229 
optional header 
data directories in, 43 
layout form, 41 

original entry point (OEP), 168 

P 

packed and encrypted code, 167 
page directory base register (PDBR), 18 
PE (portable executable), 38, 229 
plug-in syntax, 231 
flag used by, 237 
PLUGIN_UNL flag, 303 
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plugis.cfg file, 303 
pointer, 180 

popa/jmp instruction, 168 
portable executable (PE) 
binary format, 38 

and common object file format, 38 
portable executable (PE) format 
COFF-file format 
field types of, 40 
optional header layout, 40-42 
data directories 

export address table (EAT), 45 
export data table format, 44 
export directory table (EDT), 44-45 
import directory table (IDT), 46-47 
import lookup table (ILT), 46-47 
optional header, 43 
DOS file header, 31-32 
layout form of, 32 
load configuration structure, 48-49 
PRNG (pseudo-random number 
generator), 188 
processor modules, 308 
file extensions used by, 229 
procs directory, 229 
prologue procedure, 177, 182 
protocol analysis 
byte-swapping, 144-145 
ensuring message length, 150 
exit function, 143-144 
FLAG_NONE, zero definition, 145 
main parsing loop completion, 147 
processed values, 146 
role of function, 149 
single basic block packet processing, 
151-153 

three called functions, 146 
three parameters to function, 148 
use of and operator, 145 
WSARecv, 143 


protocol structure 
framing of messages 

large commercial programs, 139 
read() function, 138 
small or quickly written 
programs, 138 

reverse engineer’s access, 138 
self-similar protocols 
common API calls, reading traffic, 142 
making sense of a protocol, 141 
protocol header fields, 140 
pseudo-random number generator 
(PRNG), 188 

Q 

Qvector, 256 

R 

references to routines, 171, 172 

register window, 98 

registry key, 187 

reloading the debugger, 135 

remote buffer, 94 

remote web server, 191 

reverse engineer binaries, 200 

reverse engineering, 88, 102, 105, 166 

reversing malware, 167 

RISC architectures, 204 

routine calling, 179 

RunPlugin function, 303 
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scanff) call, 71 
ScreenEA function, 202 
scripting language, 200 
IDARub, 309 
use of, 308 
SDK 

directories of, 230 
functions, 200 
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setting of development environment 
for, 232 

version 4.9, 230 
segment descriptors, 12 
segment registers, 12 
segment selectors, 12 
semantics, 53 

SessionWriteShellThreadFn 
function, 97 
SetArray functions, 208 
shellcode, 10 

single step constructor, 180 

single stepping, executing process for, 90 

sprintfO call, 72 

spyware, 195 

stack corruption, 101 

stack frame, 20-21 

stack pointer register, 20 

stack segment register, 20 

stack window, 98 

static network detectable pattern, 190 
strcmp() call, 76 

string initialization routine, 181 
StrPassword, 76 
supval arrays, 257 
syntax highlighting editor, 212 
system management mode 
(SMM), 13 


T 

TF (trap flag), 90 

tracing, UNIX command, 91 

trap flag (TF), 90 

U 

ua_ana0 function, 251 
universal PE unpacker, 169 
unpacked version of code, 171 
user-defined code, 185 
user-submitted programs, 166 
uunp parameters, 170 

V 

variables, track of, 90 
virtualization software (VMware), 166 
Visual Studio 2003, 102 
Visual Studio 2005, 232 
Visual Studio debugger, 301 
V'Table address, 258 

W 

Website 

offensive computing, 166 
OpenRCE, 167 
Window creation, 

CreateDialogIndirectParamA(), 183 
Window initialization, sub_401AF0(), 185 
WootBot variant, 3-4 
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